Marakana Android Internals

Aleksandar (Saša) Gargenta, Ken Jones, Marko Gargenta

2012 Marakana, Inc. All rights reserved.

Last Updated: 2012-10-20

Overview

Android Stack

images/AndroidStack.svg

The purpose of this module is to explain the anatomy of the Android OS. We will walk through the Android stack, starting from the bottom and moving up:

   Android Linux Kernel Layer

images/AndroidStack-Linux.svg
Android Linux Kernel Layer

      Overview

The following are some of the changes/additions Android makes to the Linux kernel.

      Binder IPC

      Ashmem (Anonymous SHared MEMory)

      Physical memory allocator

      Wakelocks

      Early Suspend

      Alarm

      Low Memory Killer (a.k.a. Viking Killer)

      Logger

      Paranoid Network Security

      Other Kernel Changes

   Android User-Space Native Layer

images/AndroidStack-NativeLayer.svg
Android User-Space Native Layer

      Bionic Library

      Changes From BSD libc

      User-space Hardware Abstraction Layer (HAL)

      Native Daemons

      Flingers

Surface Flinger
Audio Flinger

      Function Libraries

      Dalvik

images/Stack-Dalvik-Dan.jpg
images/Dalvik-LicensePlate.jpg

   Android Application Framework Layer

images/AndroidStack-ApplicationFramework.svg
Android Application Framework Layer

      Overview

images/AndroidFooService.svg

      Activity Manager Service

      Package Manager Service

      Power Manager Service

      Alarm Manager Service

      Notification Manager Service

      Keyguard Manager Service

      Location Manager Service

      Sensor Manager Service

      Search Manager Service

      Vibrator Manager Service

      Connectivity Manager Service

      Wifi Manager Service

      Telephony Manager Service

      Input Method Manager Service

      UI Mode Manager Service

      Download Manager Service

      Storage Manager Service

      Audio Manager Service

      Window Manager Service

      Layout Inflater Manager Service

      Resource Manager Service

      Additional Manager Services

   Android Applications Layer

images/AndroidStack-Applications.svg
Android Applications Layer

      Android Built-in Applications

      Android Built-in Content Providers

      Android Built-in Input Methods

      Android Built-in Wallpapers

   Android Stack Review Questions

  1. What are the four layers of the Android Stack?

  2. What is the purpose of the Linux Kernel? How does Android use it?

  3. What’s missing in Android for it to be considered a more traditional Linux distribution?

  4. How is the Linux kernel on Android different?

  5. Name at least five Android Linux kernel extensions.

  6. What is Binder?

    1. What does Binder do and why do we need it?

    2. How is Binder exposed to the user-space?

    3. How does Binder compare to other similar mechanisms?

  7. What is Ashmem?

    1. Why do we need Ashmem?

    2. How do applications use Ashmem?

    3. How does Ashmem compare to other similar mechanisms?

  8. How does the power management on Android compare to traditional Linux distributions?

    1. What are wake locks?

    2. Where in the stack are the wake locks used?

    3. How do applications get access to wake locks?

    4. What is the purpose of early suspend feature of the Linux kernel?

  9. What is the purpose of the alarm driver?

  10. What are the two opposing objectives on Android when it comes with memory management?

    1. What is Android’s first line of defense when it comes to memory management?

    2. What is the low-memory-killer and what does it do?

    3. What are the relative levels of priority of application processes on Android?

    4. How does the low-memory-killer know who to kill and when?

    5. How can applications avoid being low-memory-killed?

  11. What is the purpose of the logger on Android?

    1. Name at least three log destinations.

    2. Where is the log information stored?

  12. What is the purpose of the CONFIG_ANDROID_PARANOID_NETWORK kernel option on Android?

  13. Name at least three sub-layers of the native layer.

  14. What is Bionic, why do we need it, and how does it differ from its alternatives?

  15. What are the two main purposes of user-space HAL on Android?

    1. How is the user-space HAL on Android exposed to the layers above it?

    2. Name at least three classes of devices exposed by user-space HAL.

  16. Name at least five native daemons and explain what they do.

    1. What is the purpose of ueventd on Android?

  17. What are the two "flingers" on Android?

  18. Name at least three "function" libraries on Android.

  19. What is the name of the Android’s media framework?

  20. What is the purpose of Dalvik on Android?

    1. How does Dalvik differ from its alternatives?

    2. What is the purpose of zygote?

    3. What was added to Dalvik in Froyo (Android 2.2)?

  21. What’s inside the Android Application Framework layer?

  22. What is the purpose of system services on Android? What value do they add?

  23. What is the purpose of managers for system services?

  24. What is the purpose of servicemanager daemon?

  25. Name at least five system services on Android.

  26. What does ActivityManagerService do?

  27. What does PackageManagerService do?

  28. What does the PowerManagerService do?

  29. What does the AlarmManagerService do?

  30. What does the KeyguardManagerService do?

  31. What does the InputMethodManagerService do?

  32. What is Android CDD and why do we need it?

  33. What’s inside an APK?

  34. What’s the difference between system and non-system apps?

  35. What are the different classes of applications that ship with Android?

  36. Name at least three "special" applications on Android?

Android Native Development Kit (NDK)

   Objectives of NDK Module

Android is put together of about equal part Java and C. So, no wonder that we need an easy way to bridge between these two totally different worlds. Java offers Java Native Interface (JNI) as a framework connecting the world of Java to the native code. Android goes a step further by packaging other useful tools and libraries into a Native Development Kit, or NDK. NDK makes developing C/C++ code that works with an Android app much simpler than if one was to do it by hand. Topics covered include:

   NDK in Action

images/Client-Native.svg
Using NDK to connect Activity to native code

   What is in NDK?

   Why NDK?

   Java Native Interface (JNI)

      JNI Overview

In this module, we’ll explore the following topics:

      JNI: What it is and why you’d care

JNI is an interface that allows Java to interact with code written in another language

Motivation for JNI:

Note JNI code is not portable!
Note

JNI can also be used to invoke Java code from within natively-written applications - such as those written in C/C++.

In fact, the java command-line utility is an example of one such application, that launches Java code in a Java Virtual Machine.

      JNI Components

      JNI By Example

  1. We start by creating a Java class with one or more native methods

    src/com/marakana/jniexamples/Hello.java:
    package com.marakana.jniexamples;

public class Hello {

    public static native void sayHi(String who, int times); // 1

    static {
        System.loadLibrary("hello"); // 2
    }

    public static void main(String[] args) {
        sayHi(args[0], Integer.parseInt(args[1])); // 3
    }
}
    1 The method sayHi(String, int) will be implemented in C/C+ in separate files, which will be compiled into a shared library.
    2 Load the shared library by its logical name. The actual name is system-dependent: libhello.so (on Linux/Unix), hello.dll (on Windows), and libhello.jnilib (Mac OSX).
    3 Here we call our native method as a regular Java method.

  2. Compile the Java code 

    $ mkdir -p bin
$ javac -d bin/ src/com/marakana/jniexamples/Hello.java

  3. Using the javah tool, we generate the C header file from the compiled com.marakana.jniexamples.Hello class: 

    $ mkdir -p jni
$ javah -jni -classpath bin -d jni com.marakana.jniexamples.Hello

  4. Observe the generated C header file:

    jni/com_marakana_jniexamples_Hello.h:
    /* DO NOT EDIT THIS FILE - it is machine generated */
#include <jni.h>
/* Header for class com_marakana_jniexamples_Hello */

#ifndef _Included_com_marakana_jniexamples_Hello
#define _Included_com_marakana_jniexamples_Hello
#ifdef __cplusplus
extern "C" {
#endif
/*
 * Class:     com_marakana_jniexamples_Hello
 * Method:    sayHi
 * Signature: (Ljava/lang/String;I)V
 */
JNIEXPORT void JNICALL Java_com_marakana_jniexamples_Hello_sayHi
  (JNIEnv *, jclass, jstring, jint);

#ifdef __cplusplus
}
#endif
#endif
    Note Method names resolve to C functions based on a pre-defined naming strategy: the prefix Java_, followed by a mangled fully-qualified class name, followed by an underscore ("_") separator, followed by a mangled method name. For overloaded native methods, two underscores ("__") followed by the mangled argument signature.

  5. Provide the C implementation:

    jni/com_marakana_jniexamples_Hello.c:
    #include "com_marakana_jniexamples_Hello.h"

JNIEXPORT void JNICALL Java_com_marakana_jniexamples_Hello_sayHi
  (JNIEnv *env, jclass clazz, jstring who, jint times) {
  const char *name = (*env)->GetStringUTFChars(env, who, NULL);
  if (name != NULL) {
    jint i;
    for (i = 0; i < times; i++) {
      printf("Hello %s\n", name);
    }
    (*env)->ReleaseStringUTFChars(env, who, name);
  }
}
    Note Most of the time, we cannot just use Java data types directly in C. For example, we have to convert java.lang.String to char * before we can effectively use it in C.
    Note This code assumes: #define NULL ((void *) 0)

  6. Compile the shared library 

    $ mkdir -p libs
$ gcc -o libs/libhello.jnilib -lc -shared \
    -I/System/Library/Frameworks/JavaVM.framework/Headers \
    jni/com_marakana_jniexamples_Hello.c
$ file libs/libhello.jnilib
libs/libhello.jnilib: Mach-O 64-bit dynamically linked shared library x86_64
    Note On Unix/Linux, compile as:
    gcc -o libs/libhello.so -lc -shared -fPIC -I$JAVA_HOME/include jni/com_marakana_jniexamples_Hello.c

  7. Run our code 

    $ java -Djava.library.path=libs -classpath bin com.marakana.jniexamples.Hello Student 5
Hello Student
Hello Student
Hello Student
Hello Student
Hello Student
    Note Instead of specifying -Djava.library.path=libs, we could have preceded our java command with export LD_LIBRARY_PATH=libs.
    Tip Common mistakes resulting in java.lang.UnsatisfiedLinkError usually come from incorrect naming of the shared library (O/S-dependent), the library not being in the search path, or wrong library being loaded by Java code.

      Native Method Arguments

      Primitive Type Mapping

      Reference Type Mapping

      Global and Local References

      Using Strings

      Arrays

Note
A note about memory

The pointer resulting from GetTypeArrayElements(…) is valid until ReleaseTypeArrayElements(…) is called (unless mode == JNI_COMMIT).

If isCopy is not NULL, then *isCopy is set to JNI_TRUE if a copy is made. When *isCopy == JNI_FALSE, the returned array is a direct pointer to the elements of the Java array, which is then pinned in memory.

Regardless of isCopy, we have to call ReleaseTypeArrayElements(…, int mode) when we are done using the native array, either to un-pin the Java array in memory when *isCopy == JNI_FALSE, or, when *isCopy == JNI_TRUE, to:

  • copy the native array over the Java array and free it (when mode == 0)
  • copy the native array over the Java array but not free it (when mode == JNI_COMMIT)
    This option assumes that we will call ReleaseTypeArrayElements(…, JNI_ABORT) at some later point
  • leave the Java array intact and free the native array (when mode == JNI_ABORT)

JNI also supports a critical version of these functions:

void * GetPrimitiveArrayCritical(JNIEnv *env, jarray array, jboolean *isCopy);
void ReleasePrimitiveArrayCritical(JNIEnv *env, jarray array, void *carray, jint mode);

The function GetPrimitiveArrayCritical(…) is similar to GetTypeArrayElements(…), except that the VM is more likely to return the pointer to the primitive array (i.e. *isCopy == JNI_FALSE is more likely). However, this comes with some caveats. The native code between GetPrimitiveArrayCritical(…) and ReleasePrimitiveArrayCritical(…) must not call other JNI functions, or make any system calls that may cause the current thread to block and wait for another Java thread. This is because the VM may temporarily suspend the garbage collection - in case it does not support memory pinning.

      Reflection

      Registering Native Methods On Load

      Exceptions

   Using NDK with C

images/NDK_Using_C.svg
NDK Process Using C

   Using NDK with C++

images/NDK_Using_CPP.svg
NDK Process Using C++

   NDK and JNI by Example

      Fibonacci - Java Native Function Prototypes

We start off by defining C function prototypes as native Java methods (wrapped in some class):

FibonacciNative/src/com/marakana/android/fibonaccinative/FibLib.java
package com.marakana.android.fibonaccinative;

import android.util.Log;

public class FibLib {
    private static final String TAG = "FibLib";

    private static long fib(long n) {
        return n <= 0 ? 0 : n == 1 ? 1 : fib(n - 1) + fib(n - 2);
    }

    // Recursive Java implementation of the Fibonacci algorithm
    // (included for comparison only)
    public static long fibJR(long n) {
        Log.d(TAG, "fibJR(" + n + ")");
        return fib(n);
    }

    // Function prototype for future native recursive implementation
    // of the Fibonacci algorithm
    public native static long fibNR(long n);


    // Iterative Java implementation of the Fibonacci algorithm
    // (included for comparison only)
    public static long fibJI(long n) {
        Log.d(TAG, "fibJI(" + n + ")");
        long previous = -1;
        long result = 1;
        for (long i = 0; i <= n; i++) {
            long sum = result + previous;
            previous = result;
            result = sum;
        }
        return result;
    }

    // Function prototype for future iterative recursive implementation
    // of the Fibonacci algorithm
    public native static long fibNI(long n);

    static {
        // as defined by LOCAL_MODULE in Android.mk
        System.loadLibrary("com_marakana_android_fibonaccinative_FibLib");
    }
}

      Fibonacci - Function Prototypes in a C Header File

We then extract our C header file with our function prototypes:

  1. On the command line, change to your project’s root directory 

    $ cd /path/to/workspace/FibonacciNative

  2. Create jni sub-directory 

    $ mkdir jni

  3. Extract the C header file from com.marakana.android.fibonaccinative.FibLib class: 

    $ javah -jni -classpath bin/classes -d jni com.marakana.android.fibonaccinative.FibLib
    Note Prior to ADT r14, compiled class files were kept directly in the bin/ directory, so in our javah command we would’ve used -classpath bin instead.

  4. Check out the resulting file:

    FibonacciNative/jni/com_marakana_android_fibonaccinative_FibLib.h
    /* DO NOT EDIT THIS FILE - it is machine generated */
#include <jni.h>
/* Header for class com_marakana_android_fibonaccinative_FibLib */

#ifndef _Included_com_marakana_android_fibonaccinative_FibLib
#define _Included_com_marakana_android_fibonaccinative_FibLib
#ifdef __cplusplus
extern "C" {
#endif
/*
 * Class:     com_marakana_android_fibonaccinative_FibLib
 * Method:    fibNR
 * Signature: (J)J
 */
JNIEXPORT jlong JNICALL Java_com_marakana_android_fibonaccinative_FibLib_fibNR
  (JNIEnv *, jclass, jlong);

/*
 * Class:     com_marakana_android_fibonaccinative_FibLib
 * Method:    fibNI
 * Signature: (J)J
 */
JNIEXPORT jlong JNICALL Java_com_marakana_android_fibonaccinative_FibLib_fibNI
  (JNIEnv *, jclass, jlong);

#ifdef __cplusplus
}
#endif
#endif
    Note The function prototype names are name-spaced to the classname they are found in.

      Fibonacci - Provide C Implementation

We provide the C implementation of com_marakana_android_fibonacci_FibLib.h header file:

FibonacciNative/jni/com_marakana_android_fibonaccinative_FibLib.c
/* Include the header file that was created via "javah -jni" command */
#include "com_marakana_android_fibonaccinative_FibLib.h"
#include <android/log.h>

/* Recursive implementation of the fibonacci algorithm (in a helper function) */
static jlong fib(jlong n) {
    return n <= 0 ? 0 : n == 1 ? 1 : fib(n - 1) + fib(n - 2);
}

/* Actual implementation of JNI-defined `fibNR` (recursive) function */
JNIEXPORT jlong JNICALL Java_com_marakana_android_fibonaccinative_FibLib_fibNR
  (JNIEnv *env, jclass clazz, jlong n) {
    __android_log_print(ANDROID_LOG_DEBUG, "FibLib.c", "fibNR(%lld)", n);
    return fib(n);
}

/* Actual implementation of JNI-defined `fibNI` (iterative) function */
JNIEXPORT jlong JNICALL Java_com_marakana_android_fibonaccinative_FibLib_fibNI
  (JNIEnv *env, jclass clazz, jlong n) {
    jlong previous = -1;
    jlong result = 1;
    jlong i;
    __android_log_print(ANDROID_LOG_DEBUG, "FibLib.c", "fibNI(%lld)", n);
    for (i = 0; i <= n; i++) {
        jlong sum = result + previous;
        previous = result;
        result = sum;
    }
    return result;
}

      Fibonacci - An Alternative Implementation (C++)

We could also use an alternative mechanism of linking native-code to managed code by pre-registering our functions. This leads to earlier detection of method-function mismatch issues, a slight performance improvement, and spares us the redundancy of the header file and the use of the javah command.

FibonacciNative/jni/com_marakana_android_fibonaccinative_FibLib.cpp
#include <jni.h>
#include <android/log.h>

namespace com_marakana_android_fibonaccinative {
    static jlong fib(jlong n) {
        return n <= 0 ? 0 : n == 1 ? 1 : fib(n - 1) + fib(n - 2);
    }

    static jlong fibNR(JNIEnv *env, jclass clazz, jlong n) {
        __android_log_print(ANDROID_LOG_DEBUG, "FibLib.c", "fibNR(%lld)", n);
        return fib(n);
    }

    static jlong fibNI(JNIEnv *env, jclass clazz, jlong n) {
        jlong previous = -1;
        jlong result = 1;
        jlong i;
        __android_log_print(ANDROID_LOG_DEBUG, "FibLib.c", "fibNI(%lld)", n);
        for (i = 0; i <= n; i++) {
            jlong sum = result + previous;
            previous = result;
            result = sum;
        }
        return result;
    }

    static JNINativeMethod method_table[] = {
            { "fibNR", "(J)J", (void *) fibNR },
            { "fibNI", "(J)J", (void *) fibNI }
    };
}

using namespace com_marakana_android_fibonaccinative;

extern "C" jint JNI_OnLoad(JavaVM* vm, void* reserved) {
    JNIEnv* env;
    if (vm->GetEnv(reinterpret_cast<void**>(&env), JNI_VERSION_1_6) != JNI_OK) {
        return JNI_ERR;
    } else {
        jclass clazz = env->FindClass("com/marakana/android/fibonaccinative/FibLib");
        if (clazz) {
            jint ret = env->RegisterNatives(clazz, method_table, sizeof(method_table) / sizeof(method_table[0]));
            env->DeleteLocalRef(clazz);
            return ret == 0 ? JNI_VERSION_1_6 : JNI_ERR;
        } else {
            return JNI_ERR;
        }
    }
}
Note Most of the Android’s JNI-based shared libraries are built using this, "alternative", approach where the functions are pre-registered.

      Fibonacci - Makefile

We need a Android.mk makefile, which will be used by NDK to compile our JNI code into a shared library:

FibonacciNative/jni/Android.mk
# Defines the root to all other relative paths
# The macro function my-dir, provided by the build system,
# specifies the path of the current directory (i.e. the
# directory containing the Android.mk file itself)
LOCAL_PATH := $(call my-dir)

# Clear all LOCAL_XXX variables with the exception of
# LOCAL_PATH (this is needed because all variables are global)
include $(CLEAR_VARS)

# List all of our C files to be compiled (header file
# dependencies are automatically computed)
LOCAL_SRC_FILES := com_marakana_android_fibonaccinative_FibLib.c

# The name of our shared module (this name will be prepended
# by lib and postfixed by .so)
LOCAL_MODULE := com_marakana_android_fibonaccinative_FibLib

# We need to tell the linker about our use of the liblog.so
LOCAL_LDLIBS += -llog

# Collects all LOCAL_XXX variables since "include $(CLEAR_VARS)"
#  and determines what to build (in this case a shared library)
include $(BUILD_SHARED_LIBRARY)
Note It’s easiest to copy the Android.mk file from another (sample) project and adjust LOCAL_SRC_FILES and LOCAL_MODULE as necessary
Note See /path/to/ndk-installation-dir/docs/ANDROID-MK.html for the complete reference of Android make files (build system)

      Fibonacci - Compile Our Shared Module

Finally, from the root of our project (i.e. FibonacciNative/), we run ndk-build to build our code into a shared library (FibonacciNative/libs/armeabi/libcom_marakana_android_fibonacci_FibLib.so):

$ ndk-build
Compile thumb  : com_marakana_android_fibonaccinative_FibLib <= com_marakana_android_fibonaccinative_FibLib.c
SharedLibrary  : libcom_marakana_android_fibonaccinative_FibLib.so
Install        : libcom_marakana_android_fibonaccinative_FibLib.so => libs/armeabi/libcom_marakana_android_fibonaccinative_FibLib.so
Note The command ndk-build comes from the NDK’s installation directory (e.g. /path/to/android-ndk-r5b), so it’s easiest if we add this directory to our PATH.
Note On Windows, older version of NDK required Cygwin (a Unix-like environment and command-line interface for Microsoft Windows) to provide "shell" (bash) and "make" (gmake) to ndk-build.
Controlling CPU Application Binary Interface (ABI)

By default, the NDK will generate machine code for the armeabi - i.e. ARMv5TE with support for Thumb-1.

In addition to ARMv5, NDK also comes with the toolchains necessary to build code for:

  • ARMv7-A (including hardware FPU/VFPv3-D16, Thumb-2, VFPv3-D32/ThumbEE, and SIMD/NEON)
  • x86 (as of r6)
  • MIPS (as of r8)

We can explicitly select ABIs using APP_ABI variable:

$ ndk-build APP_ABI=armeabi
$ ndk-build APP_ABI=armeabi-v7a
$ ndk-build APP_ABI=x86
$ ndk-build APP_ABI=mips

We can also combine them, thereby building a "fat binary":

$ ndk-build "APP_ABI=armeabi armeabi-v7a x86 mips"
$ ndk-build APP_ABI=all

Finally, we can also persist our choice of ABI, by saving APP_ABI in a Application.mk file:

jni/Application.mk:
APP_ABI := all

Each ABI-specific library, gets packaged as lib/<ABI>/lib<name>.so inside our APK.

Upon installation, the library that best matches the native ABI of the device it will execute on, will get copied to /data/data/<package>/lib/lib<name>.so.

To remove all generated binaries, run:

$ ndk-build clean
Clean: com_marakana_android_fibonaccinative_FibLib [armeabi]
Clean: stdc++ [armeabi]

      Fibonacci - Client

We can now build the "client" of our library (in this case a simple activity) to use our FibLib library.

Fibonacci - String Resources
FibonacciNative/res/values/strings.xml
<?xml version="1.0" encoding="utf-8"?>
<resources>

    <string name="hello">Get Your Fibonacci Numbers Here!</string>
    <string name="fibJR">fibJR</string>
    <string name="fibJI">fibJI</string>
    <string name="fibNR">fibNR</string>
    <string name="fibNI">fibNI</string>
    <string name="app_name">FibonacciNative</string>
    <string name="button">Get Fibonacci Result</string>

</resources>
Fibonacci - User Interface (Layout)
screens/FibonacciNativeMainLayout.png
Fibonacci Native Main Layout
FibonacciNative/res/layout/main.xml
<?xml version="1.0" encoding="utf-8"?>
<LinearLayout xmlns:android="http://schemas.android.com/apk/res/android"
    android:layout_width="fill_parent"
    android:layout_height="fill_parent"
    android:orientation="vertical" >

    <!-- This is just a simple title ("Get Your Fibonacci Here!")  -->
    <TextView
        android:layout_width="fill_parent"
        android:layout_height="wrap_content"
        android:gravity="center"
        android:text="@string/hello"
        android:textSize="25sp" />

    <!-- This is the entry box for our number "n"  -->
    <EditText
        android:id="@+id/input"
        android:layout_width="match_parent"
        android:layout_height="wrap_content"
        android:ems="10"
        android:gravity="right"
        android:inputType="number" >
        <requestFocus />
    </EditText>

    <!-- This radio group allows the user to select the fibonacci implementation type  -->
    <RadioGroup
        android:id="@+id/type"
        android:layout_width="match_parent"
        android:layout_height="wrap_content"
        android:orientation="horizontal" >

        <RadioButton
            android:id="@+id/type_fib_jr"
            android:layout_width="match_parent"
            android:layout_height="wrap_content"
            android:layout_weight="1"
            android:text="@string/fibJR" />

        <RadioButton
            android:id="@+id/type_fib_ji"
            android:layout_width="match_parent"
            android:layout_height="wrap_content"
            android:layout_weight="1"
            android:text="@string/fibJI" />

        <RadioButton
            android:id="@+id/type_fib_nr"
            android:layout_width="match_parent"
            android:layout_height="wrap_content"
            android:layout_weight="1"
            android:text="@string/fibNR" />

        <RadioButton
            android:id="@+id/type_fib_ni"
            android:layout_width="match_parent"
            android:layout_height="wrap_content"
            android:layout_weight="1"
            android:text="@string/fibNI" />
    </RadioGroup>

    <!-- This button allows the user to trigger fibonacci calculation  -->
    <Button
        android:id="@+id/button"
        android:layout_width="match_parent"
        android:layout_height="wrap_content"
        android:text="@string/button" />

    <!-- This is the output area for the fibonacci result  -->
    <TextView
        android:id="@+id/output"
        android:layout_width="match_parent"
        android:layout_height="match_parent"
        android:gravity="center"
        android:textSize="20sp" />

</LinearLayout>
Fibonacci - FibonacciActivity
FibonacciNative/src/com/marakana/android/fibonaccinative/FibonacciActivity.java
package com.marakana.android.fibonaccinative;

import android.app.Activity;
import android.app.ProgressDialog;
import android.os.AsyncTask;
import android.os.Bundle;
import android.os.SystemClock;
import android.text.TextUtils;
import android.view.View;
import android.view.View.OnClickListener;
import android.widget.Button;
import android.widget.EditText;
import android.widget.RadioGroup;
import android.widget.TextView;

public class FibonacciActivity extends Activity implements OnClickListener {

    private EditText input;

    private RadioGroup type;

    private TextView output;

    @Override
    public void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.main);
        this.input = (EditText) super.findViewById(R.id.input);
        this.type = (RadioGroup) super.findViewById(R.id.type);
        this.output = (TextView) super.findViewById(R.id.output);
        Button button = (Button) super.findViewById(R.id.button);
        button.setOnClickListener(this);
    }

    public void onClick(View view) {
        String s = this.input.getText().toString();
        if (TextUtils.isEmpty(s)) {
            return;
        }

        final ProgressDialog dialog = ProgressDialog.show(this, "",
                "Calculating...", true);
        final long n = Long.parseLong(s);
        new AsyncTask<Void, Void, String>() {

            @Override
            protected String doInBackground(Void... params) {
                long result = 0;
                long t = SystemClock.uptimeMillis();
                switch (FibonacciActivity.this.type.getCheckedRadioButtonId()) {
                case R.id.type_fib_jr:
                    result = FibLib.fibJR(n);
                    break;
                case R.id.type_fib_ji:
                    result = FibLib.fibJI(n);
                    break;
                case R.id.type_fib_nr:
                    result = FibLib.fibNR(n);
                    break;
                case R.id.type_fib_ni:
                    result = FibLib.fibNI(n);
                    break;
                }
                t = SystemClock.uptimeMillis() - t;
                return String.format("fib(%d)=%d in %d ms", n, result, t);
            }

            @Override
            protected void onPostExecute(String result) {
                dialog.dismiss();
                FibonacciActivity.this.output.setText(result);
            }
        }.execute();
    }
}

      Fibonacci - Result

screens/FibonacciNativeResult.png
Fibonacci Native Result

   NDK’s Stable APIs

The header files for NDK stable APIs are available at /path/to/ndk/platforms/<android-platform>/<arch-name>/usr/include.

      Android-specific Log Support

      ZLib Compression Library

      The OpenGL ES 1.x Library

      The OpenGL ES 2.0 Library

      The jnigraphics Library

      The OpenSL ES native audio Library

      The Android native application APIs

Caution With the exception of the libraries listed above, the native system libraries in the Android platform are not considered "stable" and may change in future platform versions. Unless our library is being built for a specific Android ROM, we should only make use of the stable libraries provided by the NDK.
Note All the header files are available under: /path/to/ndk-installation-dir/platforms/android-9/arch-arm/usr/include/
Note See /path/to/ndk-installation-dir/docs/STABLE-APIS.html for the complete reference of NDK’s stable APIs.

   Lab: NDK

The objective of this lab is to test your understanding of JNI and NDK. We will do so by adding JNI code to an existing application.

  1. Start by importing LogNative application into Eclipse

    1. Menu Bar → File → Import… → Git → Projects from Git → Next >

    2. Under Select Repository Source select URI → Next >

    3. Under Source Git Repository → Location → URI: enter https://github.com/marakana/LogNative.git → Next >

    4. Under Branch Selection, leave all branches selected (checked) → Next >

    5. Under Local Destination → Destination specify directory of your choice (e.g. ~/android/workspace/LogNative) → Next >

    6. Under Select a wizard to use for importing projects, leave Wizard for project import as Import existing projects → Next >

    7. Under Import Projects → Projects, leave LogNative as selected (checked) → Finish

      Note This project can also be downloaded as a ZIP file

  2. Examine and test your project in Eclipse

    1. Run the application on a device/emulator

    2. Enter some tag and message to log, click on the Log button and observe via adb logcat that your message get logged (assuming Java was selected)

  3. Implement com.marakana.android.lognative.LogLib.logN(int priority, String tag, String msg) in C

    1. Mark the method as native

    2. Remove its body

    3. Extract its function prototype into a C header file (hint: javah)

    4. Implement the function by taking advantage of <android/log.h> (i.e. /system/lib/liblog.so)

    5. Provide the makefile(s)

  4. Build (via ndk-build)

  5. Run your application

  6. Test by selecting Native in the UI and checking that the log tag/message shows up in adb logcat

  7. As a bonus:

    1. Throw java.lang.NullPointerException if tag or msg are null

    2. Throw java.lang.IllegalArgumentException if priority is not one of the allowed types or if tag or msg are empty

Tip Don’t forget to convert tag and msg strings from the Java format (jstring) to native format (char *) before trying to use them in int __android_log_write(int prio, const char *tag, const char *text). Be sure to free the native strings before returning from the native method. Finally, don’t forget to tell the linker about your use of the log library.

The solution is provided:

   Android NDK and JNI Review Questions

  1. Who uses JNI on Android and why?

  2. What is the NDK?

  3. What is JNI?

  4. What is the purpose of jni.h?

  5. What is the purpose of javah tool?

  6. What are the first two arguments to all native functions that model Java methods declared as native?

  7. What is jlong and why do we need it (and its cousins)?

  8. How are boolean-s modeled in native code?

  9. How are java.lang.Object-s modeled in native code?

  10. What’s the definition of NULL?

  11. What is the difference between local and global references?

  12. What is the difference between Java strings and C "strings"?

  13. What is the difference between GetStringChars and GetStringUTFChars?

  14. How do modified UTF-8 strings differ from normal UTF-8 strings?

  15. Why do we have to call ReleaseStringChars or ReleaseStringUTFChars when we are done with the C strings?

  16. Name at least five array operations?

  17. What is the purpose of mode attribute in ReleaseByteArrayElements(…, mode) (and its cousins)?

  18. What is the signature of void f(int[] a, String s, long n)?

  19. What happens when an exception is thrown by native code?

  20. What is the significance of extern "C" jint JNI_OnLoad(JavaVM* vm, void* reserved) function?

  21. What can we do with jint RegisterNatives(JNIEnv*, jclass, const JNINativeMethod*, jint) function?

  22. What does LOCAL_MODULE specify in Android.mk?

  23. How do we control the ABI when compiling NDK code?

  24. Name at least three stable NDK APIs.

   Summary of NDK Module

In this module, you learned how Android uses JNI to bridge between the world of Java and the native code. You also learned how NDK makes the process of working with JNI simpler by providing tools and framework for developing native libraries as well as packaging them with the app.

Android Binder Inter Process Communication (IPC) with AIDL

   Binder in Action

images/Client-Native.svg
Before Binder - Client connected directly to native
images/Client-Service-Native.svg
Desired Client-Service connection
images/Client-Binder-Service-Native.svg
Client-Service via Binder connection

   Why IPC?

   What is Binder?

images/BinderIPC.svg

   What is AIDL?

   Limitations of Binder

   Building a Binder-based Service and Client by Example

images/FibonacciClientServiceArchitecture.svg

      FibonacciCommon - Define AIDL Interface and Custom Types

      FibonacciService - Implement AIDL Interface and Expose It To Our Clients

      Expose our AIDL-defined Service Implementation to Clients

      FibonacciClient - Using AIDL-defined Binder-based Services

   Async-IPC via Binder by Example

      FibonacciCommon - Defining a oneway AIDL Service

      FibonacciService - Implementing our async AIDL service

      FibonacciClient - Implementing our async AIDL client

   Lab: Binder-based Service with AIDL

Create an AIDL-described ILogService that provides the following functionality:

package com.marakana.android.logservice;
public interface ILogService {
    public void log(LogMessage logMessage);
}

where LogMessage is defined as follows:

package com.marakana.android.logservice;
public class LogMessage … {
    …
    public LogMessage(int priority, String tag, String msg) {
        …
    }
    …
}

Create a simple Android client that allows the user to submit a LogMessage request to the remote ILogService running in a separate process.

You can borrow code (UI) from the basic LogNative application, which you can get:

Tip Your implementation could simply use android.util.Log.println(int priority, String tag, String msg) to do the logging.

The solution is provided:

   Review Questions

  1. Why do we need IPC on Android?

  2. What is Binder’s unit of data called?

  3. What is AIDL and when do we use it?

  4. Is the use of AIDL required when consuming or exposing bound services?

  5. Name at least five supported method parameter/return data types that we use in AIDL.

  6. Name at least three types that are not supported.

  7. What is android.os.Bundle?

  8. What is android.os.Parcelable and how do we implement it?

  9. What do we have to do with android.os.Parcelable classes before we can use them in AIDL interfaces?

  10. What is android.os.IBinder?

  11. What is special about file descriptors?

  12. What is the purpose of the directional flag?

  13. What is the purpose of the aidl tool?

  14. What is the purpose of the Stub and the Proxy?

  15. What is the Android library project and when do we use them?

  16. What is the purpose of android.app.Service in the context of bound services?

  17. How do we expose a bound service to other applications?

  18. What is the thread context of an binder service request?

  19. How and when do we bind to non-system services?

  20. What is the life-cycle dependency between bound services and their clients?

  21. What is the purpose of the oneway keyword and when do we use it?

  22. What do we have to worry about when handling call-backs from remote services?

Android Security Essentials

   Android Security Architecture

images/AndroidProcessSnapshot-NoPermissions.svg

   Application Signing

      Platform Keys

   User IDs

   File Access

   Permissions

      Using Permissions

      Top Ten Bad Permissions (on Google Play)

Using the following permissions will significantly lower the likelihood for an Android app/game to be featured in Google Play (from Google I/O 2012):

  1. android.permission.SEND_SMS and android.permission.RECEIVE_SMS

  2. android.permission.SYSTEM_ALERT_WINDOW

  3. com.android.browser.permission.READ_HISTORY_BOOKMARKS and com.android.browser.permission.WRITE_HISTORY_BOOKMARKS

  4. android.permission.READ_CONTACTS, android.permission.WRITE_CONTACTS, android.permission.READ_CALENDAR, android.permission.WRITE_CALENDAR

  5. android.permission.CALL_PHONE

  6. android.permission.READ_LOGS

  7. android.permission.ACCESS_FINE_LOCATION

  8. android.permission.GET_TASKS

  9. android.permission.RECEIVE_BOOT_COMPLETED

  10. android.permission.CHANGE_WIFI_STATE

      Avoid Using Permissions (When You Can)

      Permission Enforcement

There are a number of trigger points for security/permission checks:

      Kernel / File-system Permission Enforcement

      UID-based Permission Enforcement

      Static Permission Enforcement

      Dynamic Permission Enforcement

images/AndroidSecurityCallFlow.svg

      Custom Permissions

Before we can enforce our own permissions, we have to declare them using one or more <permission> in

AndroidManifest.xml
<manifest xmlns:android="http://schemas.android.com/apk/res/android" package="com.marakana.android.myapp" >
  <permission
    android:name="com.example.app.DO_X"
    android:label="@string/do_x_label"
    android:description="@string/do_x_desc"
    android:permissionGroup="android.permission-group.PERSONAL_INFO"
    android:protectionLevel="dangerous" />
  …
</manifest>

      Adding Custom Permissions Dynamically

      Custom Permissions by Example

Static Permission Enforcement

Here, we want to restrict access to the com.marakana.android.fibonacciservice.FibonacciService to applications (i.e. clients) that hold USE_FIBONACCI_SERVICE custom permission

  1. We start by by creating a custom permission group (making sure that we name-space it):

    FibonacciService/res/values/strings.xml:
    <?xml version="1.0" encoding="utf-8"?>
<resources>
    …
    <string name="fibonacci_permissions_group_label">Fibonacci Permissions</string>
    …
</resources>
    FibonacciService/AndroidManifest.xml:
    <?xml version="1.0" encoding="utf-8"?>
<manifest …>
    …
    <permission-group
        android:name="com.marakana.android.fibonacciservice.FIBONACCI_PERMISSIONS"
        android:label="@string/fibonacci_permissions_group_label" />
    …
</manifest>
    Note This permission group is optional - as we could instead use one of the already provided groups

  2. Next, we create a custom permission (again, making sure that we name-space it), while taking advantage of our newly-created permission group:

    FibonacciService/res/values/strings.xml:
    <?xml version="1.0" encoding="utf-8"?>
<resources>
    …
    <string name="use_fibonacci_service_permission_label">use fibonacci service</string>
    <string name="use_fibonacci_service_permission_description">
      applications with this permissions get fibonacci results for free
    </string>
    …
</resources>
    FibonacciService/AndroidManifest.xml:
    <?xml version="1.0" encoding="utf-8"?>
<manifest …>
    …
    <permission-group …/>
    <permission
        android:name="com.marakana.android.fibonacciservice.USE_FIBONACCI_SERVICE"
        android:description="@string/use_fibonacci_service_permission_description"
        android:label="@string/use_fibonacci_service_permission_label"
        android:permissionGroup="com.marakana.android.fibonacciservice.FIBONACCI_PERMISSIONS"
        android:protectionLevel="dangerous" />
    …
</manifest>

  3. Now we can statically require the permission on our FibonacciService service:

    FibonacciService/AndroidManifest.xml:
    <?xml version="1.0" encoding="utf-8"?>
<manifest …>
    …
    <permission-group …/>
    <permission …/>
    <application …>
        <service
            android:name=".FibonacciService"
            android:permission="com.marakana.android.fibonacciservice.USE_FIBONACCI_SERVICE" >
            …
        </service>
    </application>
    …
</manifest>

  4. If we now re-run the FibonacciService and re-run the FibonacciClient, we will notice that the client will fail to launch and adb logcat will show something like: 

    …
W/ActivityManager(   85): Permission Denial: Accessing service ComponentInfo{com.marakana.android.fibonacciservice/com.marakana.android.fibonacciservice.FibonacciService} from pid=540, uid=10043 requires com.marakana.android.fibonacciservice.USE_FIBONACCI_SERVICE
D/AndroidRuntime(  540): Shutting down VM
W/dalvikvm(  540): threadid=1: thread exiting with uncaught exception (group=0x409c01f8)
E/AndroidRuntime(  540): FATAL EXCEPTION: main
E/AndroidRuntime(  540): java.lang.RuntimeException: Unable to resume activity {com.marakana.android.fibonacciclient/com.marakana.android.fibonacciclient.FibonacciActivity}: java.lang.SecurityException: Not allowed to bind to service Intent { act=com.marakana.android.fibonaccicommon.IFibonacciService }
E/AndroidRuntime(  540):    at android.app.ActivityThread.performResumeActivity(ActivityThread.java:2444)
…
E/AndroidRuntime(  540):    at dalvik.system.NativeStart.main(Native Method)
E/AndroidRuntime(  540): Caused by: java.lang.SecurityException: Not allowed to bind to service Intent { act=com.marakana.android.fibonaccicommon.IFibonacciService }
E/AndroidRuntime(  540):    at android.app.ContextImpl.bindService(ContextImpl.java:1135)
E/AndroidRuntime(  540):    at android.content.ContextWrapper.bindService(ContextWrapper.java:370)
E/AndroidRuntime(  540):    at com.marakana.android.fibonacciclient.FibonacciActivity.onResume(FibonacciActivity.java:65)
…
W/ActivityManager(   85):   Force finishing activity com.marakana.android.fibonacciclient/.FibonacciActivity
…

  5. Finally, we can give FibonacciClient a fighting chance by allowing it to use the USE_FIBONACCI_SERVICE permission:

    FibonacciClient/AndroidManifest.xml:
    <?xml version="1.0" encoding="utf-8"?>
<manifest …>
    …
    <uses-permission android:name="com.marakana.android.fibonacciservice.USE_FIBONACCI_SERVICE"/>
    …
</manifest>

  6. We can now observe that our client is again able to use the service

  7. In the Emulator, if we go to Home → Menu → Manage apps → Fibonacci Client → PERMISSIONS, we should see the Fibonacci Permissions group and under it, use fibonacci service permission

Dynamic Permission Enforcement

Here, we want to restrict access to the com.marakana.android.fibonacciservice.IFibonacciServiceImpl's recursive operations (fibJR(long n) and fibNR(long n)) for n > 10 to applications (i.e. clients) that hold USE_SLOW_FIBONACCI_SERVICE custom permission

  1. Like before, we start off by creating a custom permission:

    FibonacciService/res/values/strings.xml:
    <?xml version="1.0" encoding="utf-8"?>
<resources>
    …
    <string name="use_slow_fibonacci_service_permission_label">
        use slow fibonacci service operations
   </string>
    <string name="use_slow_fibonacci_service_permission_description">
        applications with this permissions can melt the CPU and drain the battery
        by using slow fibonacci operations
    </string>
    …
</resources>
    FibonacciService/AndroidManifest.xml:
    <?xml version="1.0" encoding="utf-8"?>
<manifest …>
    …
    <permission-group …/>
    <permission …/>
    <permission
        android:name="com.marakana.android.fibonacciservice.USE_SLOW_FIBONACCI_SERVICE"
        android:description="@string/use_slow_fibonacci_service_permission_description"
        android:label="@string/use_slow_fibonacci_service_permission_label"
        android:permissionGroup="com.marakana.android.fibonacciservice.FIBONACCI_PERMISSIONS"
        android:protectionLevel="dangerous" />
    …
</manifest>

  2. Next, we update our IFibonacciServiceImpl to enforce this permission dynamically - via a android.content.Context that get expect to get through the constructor:

    FibonacciService/src/com/marakana/android/fibonacciservice/IFibonacciServiceImpl.java:
    package com.marakana.android.fibonacciservice;

import android.content.Context;
…
public class IFibonacciServiceImpl extends IFibonacciService.Stub {
    …
    private final Context context;

    public IFibonacciServiceImpl(Context context) {
        this.context = context;
    }

    private long checkN(long n) {
        if (n > 10) {
            this.context.enforceCallingOrSelfPermission(
                    Manifest.permission.USE_SLOW_FIBONACCI_SERVICE, "Go away!");
        }
        return n;
    }
    …
    public long fibJR(long n) {
        …
        return FibLib.fibJR(this.checkN(n));
    }
    …
    public long fibNR(long n) {
        …
        return FibLib.fibNR(this.checkN(n));
    }
    …
}

  3. We have to update FibonacciService to invoke the new IFibonacciServiceImpl's constructor:

    FibonacciService/src/com/marakana/android/fibonacciservice/FibonacciService.java:
    …
public class FibonacciService extends Service {
    …
    @Override
    public void onCreate() {
        …
        this.service = new IFibonacciServiceImpl(super.getApplicationContext());
        …
    }
    …
}

  4. If we now re-run the FibonacciService and re-run the FibonacciClient for a recursive operation with n > 10, we will notice that the client will fail and adb logcat will show something like: 

    …
D/IFibonacciServiceImpl(  617): fib(15, RECURSIVE_NATIVE)
D/IFibonacciServiceImpl(  617): fibNR(15)
W/dalvikvm(  604): threadid=11: thread exiting with uncaught exception (group=0x409c01f8)
E/AndroidRuntime(  604): FATAL EXCEPTION: AsyncTask #1
E/AndroidRuntime(  604): java.lang.RuntimeException: An error occured while executing doInBackground()
…
E/AndroidRuntime(  604):    at java.lang.Thread.run(Thread.java:856)
E/AndroidRuntime(  604): Caused by: java.lang.SecurityException: Go away!: Neither user 10043 nor current process has com.marakana.android.fibonacciservice.USE_SLOW_FIBONACCI_SERVICE.
…

  5. Finally, we can allow FibonacciClient to melt our CPU and drain our battery by allowing it to use the USE_SLOW_FIBONACCI_SERVICE permission:

    FibonacciClient/AndroidManifest.xml:
    <?xml version="1.0" encoding="utf-8"?>
<manifest …>
    …
    <uses-permission android:name="com.marakana.android.fibonacciservice.USE_SLOW_FIBONACCI_SERVICE"/>
    …
</manifest>

  6. We can now observe that our client is again able to use recursive fibonacci operations even for n > 10

  7. In the Emulator, if we go to Home → Menu → Manage apps → Fibonacci Client → PERMISSIONS → Fibonacci Permissions, we should see both use fibonacci service and use slow fibonacci service operations permissions

      Lab: Custom Permissions

      ContentProvider URI Permissions

      Public vs. Private Components

      Intent Broadcast Permissions

      Pending Intents

   Review Questions

  1. What is the basic philosophy of the Android security model?

  2. What does Android do to implement this philosophy?

  3. How and when are permissions used, granted, and enforced?

  4. What is a confused deputy attack?

  5. What is a collusion attack?

  6. Which part of Android is ultimately responsible for providing the system security?

  7. How does native code differ from Java code when it comes to its security context?

  8. Why do we sign applications?

  9. What are the four platform keys?

  10. What is the relationship between apps and Linux user IDs?

  11. When are the user IDs assigned to apps?

  12. Where are the user ID assignments recorded?

  13. What is the relationship between application user IDs and its file-system resources?

  14. Where is the home of an app with the package name com.foo.bar?

  15. How do apps share a user IDs?

  16. How do apps share a process? Provide at least one use-case for this.

  17. Name at least five (file) system volumes (think mount points).

  18. What is the purpose of /data/dalvik-cache/?

  19. What is the purpose of run-as command?

  20. How do you use permissions?

  21. What happens if you don’t use a permission but you attempt to use a restricted API?

  22. How are the permissions enforced?

  23. How do bound services enforce their permissions? What makes that possible?

  24. What are the attributes of the <permission … /> tag?

  25. What are the possible values for protectionLevel attribute?

  26. What does protectedLevel=signatureOrSystem mean?

  27. What is the value of grouping permissions?

  28. What is the significance of android:grantUriPermission attribute on content providers?

  29. What is the significance android:exported attribute on Android components?

  30. How do you protected unauthorized applications from receiving your broadcasts (intents)?

  31. What’s the worry with pending intents?

Building Android From Source

   Why Build Android From Source?

   Setting up the Build Environment

   Downloading the Source Tree

   Android Source Code Structure

   Android Build System

   Initializing the Build Environment

   Choosing the Build Target

Note Additional CodeName-BuildType combinations may be available based on what build/envsetup.sh finds (usually in the device/ folder)
Note Building for real hardware usually requires that we get proprietary binaries (mostly user-space HAL). For Nexus and other Google-supported devices, we can go to http://code.google.com/android/nexus/drivers.html, which allows us to download scripts, which in turn extract binaries from the connected devices (via adb pull) into vendor/ directory tree structure.

   Compiling Android

      Makefile targets

Android build system Makefile (actually build/core/main.mk) includes some of the following targets:

Make targets Description

droid

The default target (build the full system)

clean

Equivalent to rm -rf out/ (same as make clobber)

installclean

Deletes all of the files that change between different build types, like make user vs. make sdk

dataclean

Delete files in the staging and emulator data partitions: data/*, data-qemu/*, and userdata-qemu.img

snod

Quickly rebuild the system image from built packages

offline-sdk-docs

Generate the HTML for the developer SDK docs

doc-comment-check-docs

Check HTML doc links and validity, without generating HTML

libandroid_runtime

All the JNI framework stuff

framework

All the java framework stuff

services

The system server (Java) and friends

sdk

Build the Android SDK (tools)

help

Display a help message listing some of these targets

modules

Display a list of modules that can be built (where each module name is specified by LOCAL_MODULE)

<module-name>

Make just a specific module (same as cd module/dir && mm)

clean-<module-name>

Clean just a specific module

otacerts

OTA keys that are used to verify OTA packages

…

Etc.

   Examining the Built Images

   Running Custom Android Build on Emulator

   Running Custom Android Build on Real Hardware

   Building the Linux Kernel

Note It is easiest to build the Linux kernel on a Linux OS. While other host OSs can also be used, they are not trivial to setup.

   Getting the Kernel

$ git clone https://android.googlesource.com/kernel/common.git
$ git clone https://android.googlesource.com/kernel/goldfish.git
$ git clone https://android.googlesource.com/kernel/msm.git
$ git clone https://android.googlesource.com/kernel/omap.git
$ git clone https://android.googlesource.com/kernel/samsung.git
$ git clone https://android.googlesource.com/kernel/tegra.git

      Building Kernel for the Emulator (Goldfish)

  1. Start the emulator

  2. Get the existing kernel version from /proc/version (since uname does not exist on Android) 

    $ adb shell cat /proc/version
Linux version 2.6.29-g46b05b2 (vchtchetkine@vc-irv.irv.corp.google.com) (gcc version 4.4.3 (GCC) ) #28 Thu Nov 17 06:39:36 PST 2011

  3. Get the corresponding kernel version (here, we are getting the kernel for goldfish, the emulator) 

    $ git clone https://android.googlesource.com/kernel/goldfish.git
$ cd goldfish/
$ git branch -a
$ git checkout -t remotes/origin/android-goldfish-2.6.29
    Note Alternatively, we could directly clone the goldfish 2.6.29 branch
    $ git clone https://android.googlesource.com/kernel/goldfish.git -b android-goldfish-2.6.29

  4. Specify the target architecture and cross compiler 

    $ export ARCH=arm
$ export CROSS_COMPILE=/path/to/android-src/prebuilt/linux-x86/toolchain/arm-eabi-4.4.3/bin/arm-eabi-
    Caution Android’s own build system uses ARCH and CROSS_COMPILE env vars, so these need to be cleared before building AOSP.
    As an alternative, these can be also passed directly to the make commands: $ make … ARCH=… CROSS_COMPILE=…

  5. Get the existing kernel configuration file (with all of the Android/emulator specific options) by pulling it from the running emulator: 

    $ adb pull /proc/config.gz .
$ gunzip config.gz
$ mv config .config
    Note As an alternative, we could generate the Goldfish .config file (even without the emulator) by running:
    $ make goldfish_armv7_defconfig ARCH=arm

  6. You can optionally take a look at the existing configuration options and make changes as desired 

    $ make menuconfig ARCH=arm
    • For example, you could set your General setup → Local version - append to kernel release to something like -marakana-example

  7. Now we are ready to compile 

    $ make

  8. The resulting kernel will be compressed to arch/arm/boot/zImage

  9. Run the emulator with our new kernel 

    $ $ANDROID_HOST_OUT/bin/emulator -kernel /path/to/common/arch/arm/boot/zImage

  10. And the result is

    screens/CustomKernel.png

   Review Questions

  1. What is repo?

  2. What Android’s build system based on?

  3. What is the first thing you need to do before you can build Android (after you get the source)?

  4. What is the unit of build?

  5. What are the different build types and how do they differ?

  6. What is the purpose of ccache?

  7. What is the end result of a build?

  8. What’s inside recovery.img?

  9. What is the purpose of fastboot?

  10. What is the purpose of CROSS_COMPILE variable and what do you set it to when compiling the kernel?

Android Startup

images/AndroidProcessSnapshot.svg

   Bootloading the Kernel

  1. On power-up, CPU is uninitialized - wait for stable power

  2. Execute Boot ROM (hardwired into CPU)

    1. Locate the first-stage boot loader

    2. Load the first-stage boot loader into internal RAM

    3. Jump to first-stage boot loader’s memory location to execute it

  3. First-stage boot loader runs

    1. Detect and initialize external RAM

    2. Locate the second-stage boot loader

    3. Load the second-stage boot loader into external RAM

    4. Jump to the second-stage boot loader’s memory location to execute it

  4. Second-stage boot loader runs

    1. Setup file systems (typically on Flash media)

    2. Optionally setup display, network, additional memory, and other devices

    3. Enable additional CPU features

    4. Enable low-level memory protection

    5. Optionally load security protections (e.g. ROM validation code)

    6. Locate Linux Kernel

    7. Load Linux Kernel into RAM

    8. Place Linux Kernel boot parameters into memory so that kernel knows what to run upon startup

    9. Jump to Linux Kernel memory address to run it

  5. Linux Kernel runs

    1. Build a table in RAM describing the layout of the physical memory

    2. Initialize and setup input devices

    3. Initialize and setup disk (typically MTD) controllers and map available block devices in RAM

    4. Initialize Advanced Power Management (APM) support

    5. Initialize interrupt handlers: Interrupt Descriptor Table (IDT), Global Descriptor Table (GDT), and Programmable Interrupt Controllers (PIC)

    6. Reset the floating-point unit (FPU)

    7. Switch from real to protected mode (i.e. enable memory protection)

    8. Initialize segmentation registers and a provisional stack

    9. Zero uninitialized memory

    10. Decompress the kernel image

    11. Initialize provisional kernel page tables and enable paging

    12. Setup kernel mode stack for process 0

    13. Fill the IDT with null interrupt handlers

    14. Initialize the first page frame with system parameters

    15. Identify the CPU model

    16. Initialize registers with the addresses of the GDT and IDT

    17. Initialize and start the kernel

      1. Scheduler

      2. Memory zones

      3. Buddy system allocator

      4. IDT

      5. SoftIRQs

      6. Date and Time

      7. Slab allocator

      8. …

    18. Create process 1 (/init) and run it

   Android’s init Startup

images/StartupWalkthru.svg

   Zygote Startup

  1. Zygote starts from /init.rc 

    service zygote /system/bin/app_process -Xzygote /system/bin --zygote --start-system-server
    socket zygote stream 666
    onrestart write /sys/android_power/request_state wake
    onrestart write /sys/power/state on
    onrestart restart media
    onrestart restart netd

  2. This translates to frameworks/base/cmds/app_process/app_main.cpp:main()

  3. The command app_process then launches frameworks/base/core/java/com/android/internal/os/ZygoteInit.java:main() in a Dalvik VM via frameworks/base/core/jni/AndroidRuntime.cpp:start()

  4. ZygoteInit.main() then

    1. Registers for zygote socket

    2. Pre-loads classes defined in frameworks/base/preloaded-classes (1800+)

    3. Pre-loads resources preloaded_drawables and preloaded_color_state_lists from frameworks/base/core/res/res/values/arrays.xml

    4. Runs garbage collector (to clean the memory as much as possible)

    5. Forks itself to start system_server

    6. Starts listening for requests to fork itself for other apps

   System Server Startup

  1. When Zygote forks itself to launch the system_server process (in ZygoteInit.java:startSystemServer()), it executes frameworks/base/services/java/com/android/server/SystemServer:java.main()

  2. The SystemServer:java.main() method loads android_servers JNI lib from frameworks/base/services/jni and invokes init1() native method

  3. Before init1() runs, the JNI loader first runs frameworks/base/services/jni/onload.cpp:JNI_OnLoad(), which registers native services - to be used as JNI counterparts to Java-based service manager loaded later

  4. Now frameworks/base/services/jni/com_android_server_SystemServer.cpp:init1() is invoked, which simply wraps a call to frameworks/base/cmds/system_server/library/system_init.cpp:system_init()

  5. The system_init.cpp:system_init() function

    1. First starts native services (some optionally):

      1. frameworks/base/services/surfaceflinger/SurfaceFlinger.cpp

      2. frameworks/base/services/sensorservice/SensorService.cpp

      3. frameworks/base/services/audioflinger/AudioFlinger.cpp

      4. frameworks/base/media/libmediaplayerservice/MediaPlayerService.cpp

      5. frameworks/base/camera/libcameraservice/CameraService.cpp

      6. frameworks/base/services/audioflinger/AudioPolicyService.cpp

    2. Then goes back to frameworks/base/services/java/com/android/server/SystemServer.java:init2(), again via frameworks/base/core/jni/AndroidRuntime.cpp:start() JNI call

  6. The SystemServer.java:init2() method then starts Java service managers in a separate thread (ServerThread), readies them, and registers each one with frameworks/base/core/java/android/os/ServiceManager:addService() (which in turn delegates to to ServiceManagerNative.java, which effectively talks to servicemanager daemon previously started by init)

    1. frameworks/base/services/java/com/android/server/PowerManagerService.java

    2. frameworks/base/services/java/com/android/server/am/ActivityManagerService.java

    3. frameworks/base/services/java/com/android/server/TelephonyRegistry.java

    4. frameworks/base/services/java/com/android/server/PackageManagerService.java

    5. frameworks/base/services/java/com/android/server/BatteryService.java

    6. frameworks/base/services/java/com/android/server/VibratorService.java

    7. etc

  7. Finally frameworks/base/services/java/com/android/server/am/ActivityManagerService.java:finishBooting() sets sys.boot_completed=1 and sends out

    1. a broadcast intent with android.intent.action.PRE_BOOT_COMPLETED action (to give apps a chance to reach to boot upgrades)

    2. an activity intent with android.intent.category.HOME category to launch the Home (or Launcher) application

    3. a broadcast intent with android.intent.action.BOOT_COMPLETED action, which launches applications subscribed to this intent (while using android.permission.RECEIVE_BOOT_COMPLETED)

   Review Questions

  1. What is the purpose of the second-stage bootloader?

  2. What is the first thing that runs (in the user-space) after the kernel is initialized?

  3. Where does this process (with PID==1) get its initialization instructions from?

  4. What are actions (of process with PID==1)?

  5. Name at least three triggers and three commands for actions.

  6. What are services (of process with PID==1)?

  7. How are services launched?

  8. What’s the difference between critical and non-critical services?

  9. Name at least five services started by process with PID==1.

  10. What is the purpose of ueventd process?

  11. What is the purpose of zygote process and what does it do upon initialization?

  12. What is the purpose of system_server process and what does it do upon initialization?

  13. Who is responsible for launching the Home application and how is it done?

  14. How do applications that want to start on boot get launched?

Android Subsystems

The objective of this module is to explain the inter-workings of various Android subsystems. There are close to sixty various services in ICS release. In this module, we’ve hand-picked some of the more common ones. By the end of the module, you should start to understand some common traits of Android subsystem architectures, such as use of the Binder for inter-process communication, and use of JNI for Java-C interaction.

Android services are the key to exposing lower level functionality of the hardware and the Linux kernel to the high level Android apps. Understanding how they work creates the opportunity to customize and extend their behavior, or add another service altogether.

$ adb shell service list
Found 56 services:
0   phone: [com.android.internal.telephony.ITelephony]
1   iphonesubinfo: [com.android.internal.telephony.IPhoneSubInfo]
2   simphonebook: [com.android.internal.telephony.IIccPhoneBook]
3   isms: [com.android.internal.telephony.ISms]
4   samplingprofiler: []
5   diskstats: []
6   appwidget: [com.android.internal.appwidget.IAppWidgetService]
7   backup: [android.app.backup.IBackupManager]
8   uimode: [android.app.IUiModeManager]
9   usb: [android.hardware.usb.IUsbManager]
10  audio: [android.media.IAudioService]
11  wallpaper: [android.app.IWallpaperManager]
12  dropbox: [com.android.internal.os.IDropBoxManagerService]
13  search: [android.app.ISearchManager]
14  country_detector: [android.location.ICountryDetector]
15  location: [android.location.ILocationManager]
16  devicestoragemonitor: []
17  notification: [android.app.INotificationManager]
18  mount: [IMountService]
19  throttle: [android.net.IThrottleManager]
20  connectivity: [android.net.IConnectivityManager]
21  wifi: [android.net.wifi.IWifiManager]
22  wifip2p: [android.net.wifi.p2p.IWifiP2pManager]
23  netpolicy: [android.net.INetworkPolicyManager]
24  netstats: [android.net.INetworkStatsService]
25  textservices: [com.android.internal.textservice.ITextServicesManager]
26  network_management: [android.os.INetworkManagementService]
27  clipboard: [android.content.IClipboard]
28  statusbar: [com.android.internal.statusbar.IStatusBarService]
29  device_policy: [android.app.admin.IDevicePolicyManager]
30  accessibility: [android.view.accessibility.IAccessibilityManager]
31  input_method: [com.android.internal.view.IInputMethodManager]
32  window: [android.view.IWindowManager]
33  alarm: [android.app.IAlarmManager]
34  vibrator: [android.os.IVibratorService]
35  battery: []
36  hardware: [android.os.IHardwareService]
37  content: [android.content.IContentService]
38  account: [android.accounts.IAccountManager]
39  permission: [android.os.IPermissionController]
40  cpuinfo: []
41  gfxinfo: []
42  meminfo: []
43  activity: [android.app.IActivityManager]
44  package: [android.content.pm.IPackageManager]
45  telephony.registry: [com.android.internal.telephony.ITelephonyRegistry]
46  usagestats: [com.android.internal.app.IUsageStats]
47  batteryinfo: [com.android.internal.app.IBatteryStats]
48  power: [android.os.IPowerManager]
49  entropy: []
50  sensorservice: [android.gui.SensorServer]
51  media.audio_policy: [android.media.IAudioPolicyService]
52  media.camera: [android.hardware.IAudioPolicyServiceService]
53  media.player: [android.media.IMediaPlayerService]
54  media.audio_flinger: [android.media.IAudioFlinger]
55  SurfaceFlinger: [android.ui.ISurfaceComposer]

   Vibrator on Android

images/VibratorArchitecture.svg
Vibrator Service Architecture

   Power on Android

images/PowerManagerServiceArchitecture.svg
Power Service Architecture

      Resources

   Alarm on Android

images/AlarmManagerServiceArchitecture.svg
Alarm Service Architecture

   Package Management on Android

images/PackageManagerServiceArchitecture.svg
Package Manager Service Architecture

   WiFi on Android

Wifi Service exposes WiFi functionality of the underlying system to the application layer via WifiManager class. The key differentiation between wifi stack and some other ones is that the wifi stack primarily uses the wpa_supplicant to talk to the Wifi driver.

images/WifiArchitecture.svg
Wifi Service Architecture

      Resources

   Location on Android

images/LocationManagerServiceArchitecture.svg
Location Service Architecture

   Audio on Android

images/AudioPlaybackServiceArchitecture.svg
Audio Architecture (playing audio)

images/AudioManagerServiceArchitecture.svg

      Resources

   Media on Android

      Introduction

The media framework are the APIs and libraries used for controlling playback and recording of video/audio. Since everything involving video and audio require a lot of computing power, mobile devices usually use a lot of special-purpose hardware for this, compared to desktop computers where there normally is enough raw power to run most/everything in software. Full-software solutions normally are more flexible and portable, but obviously require more processor power (which also might give a higher power consumption).

Normally only the highest level APIs are public, to allow for flexibility in the implementation internally.

      Typical stack of function calls

  1. java: android.media.MediaPlayer
    frameworks/base/media/java/android/media/MediaPlayer.java

  2. JNI: frameworks/base/media/jni/android_media_MediaPlayer.cpp
    Quite straight mapping of java native functions to the C++ MediaPlayer class

  3. C++: MediaPlayer
    frameworks/base/media/libmedia/mediaplayer.cpp

  4. IMediaPlayer, IMediaPlayerService
    frameworks/base/include/media/IMediaPlayer.h
    frameworks/base/include/media/IMediaPlayerService.h

  5. BpMediaPlayer: Binder client/proxy interface
    frameworks/base/media/libmedia/IMediaPlayer.cpp

  6. BnMediaPlayer: Binder implementation interface (running in the media server)

  7. BnMediaPlayer implemented by MediaPlayerService::Client
    frameworks/base/libmediaplayservice/MediaPlayerService.cpp

  8. MediaPlayerService backed by different implementations:

    • PVPlayer (old OpenCORE based player, phased out in gingerbread, removed later)
    • StagefrightPlayer
      frameworks/base/media/libmediaplayerservice/StagefrightPlayer.cpp
      • AwesomePlayer
        frameworks/base/media/libstagefright/AwesomePlayer.cpp
      • OMXCodec
        frameworks/base/media/libstagefright/OMXCodec.cpp
        Pull based stagefright element, wrapping IOMX
      • AudioPlayer
        frameworks/base/media/libstagefright/AudioPlayer.cpp
    • NuPlayer (new in ICS/Honeycomb only for HTTP Live Streaming)
      frameworks/base/media/libmediaplayerservice/nuplayer/NuPlayerDriver.cpp
      • ACodec
        frameworks/base/media/libstagefright/ACodec.cpp message passing based frontend on top of IOMX

  9. MediaPlayerService::AudioOutput
    frameworks/base/libmediaplayservice/MediaPlayerService.cpp

  10. AudioTrack
    frameworks/base/media/libmedia/AudioTrack.cpp

  11. IAudioTrack

    • BpAudioTrack, BnAudioTrack

  12. AudioFlinger::TrackHandle
    frameworks/base/services/audioflinger/AudioFlinger.cpp

The actual decoding/encoding in the OMXCodec and ACodec class is implemented via the IOMX interface:

The applications may also use OpenSL ES for playing back audio this internally uses OMXCodec from stagefright for decoding of the audio. (Code for this is in system/media/opensles/libopensles/android SfPlayer.cpp in gingerbread, in system/media/wilhelm/src/android/android AudioSfDecoder.cpp in ICS.) In ICS, there’s also a an OpenMAX AL API that can play back video, using the IMediaPlayerService interface.

      The media server process

Most of the APIs above in section 2 can either run in the calling application processes, or in the media server. Each of the Binder APIs (IAPI/BpAPI/BnAPI, such as IOMX, BpOMX, BnOMX) can proxy calls into the media server. The caller only gets an instance of the generic interface, e.g. IMediaPlayerm and does not know whether this is a proxy for calling the same interface in another process (BpOMX) or the actual implementation running within the same process (BnOMX).

Thanks to this, the whole concept of some parts of the API running within a separate process is mostly transparent when working with the APIs.

      Stagefright

Intro

Stagefright is the library containing most of the implementation of the media framework.

Stagefright is a relatively recent development within android, replacing OpenCORE. It first appeared in Eclair, was made default player for all video (except for RTSP streaming which was still handled by OpenCORE) in Froyo, and in Gingerbread it had gained RTSP streaming support and was mature enough to be used for video recording, too, so OpenCORE was removed in Gingerbread. Since then, it has still evolved a bit.

Stagefright is mainly a generic pipeline kind of framework, similar to GStreamer, but much smaller and simpler. (It is purpose-written from scratch to fill the exact needs of Android, while OpenCORE was an already existing framework that PacketVideo provided at the time. Stagefright is a few orders of magnitude smaller than OpenCORE while still fulfilling all the needs Android has.)

Example

Being a pipeline framework means that it consists of individual elements that form a pipeline, where each element either can produce data itself, or take input data from another element. By chaining such elements together, one can produce a pipeline that does the intended thing. E.g., for playing back video from a mp4 file on the phone, the following elements are hooked up together:

The AwesomePlayer creates the right MediaExtractor in finishSetDataSource_1, setting up the tracks in setDataSource l, and later creates the OMXCodec object for each of the tracks. The audio OMXCodec object is given as source object to AudioPlayer, which then reads data from it.

GStreamer has the same element/pipeline design, but is much more flexible supporting a number of modes of operation and a lot more elements, while Stagefright is simpler and smaller, since it only does exactly what is needed in Android.

Pipeline

The pipeline in Stagefright is pull-based, meaning that each element deriving from the base class MediaSource implements a method read(), which when called will block until the MediaSource element has produced one MediaBuffer which is returned. When the MediaSource subclass object is created, it is normally given an upstream MediaSource object to read from, e.g. for OMXCodec, the MediaSource is given as parameter to OMXCodec::Create().

For OMXCodec, the read() method internally reads a packet of compressed data from the source (which might e.g. be MPEG4Source within MPEG4Extractor in the case above), passes the packet to the actual OpenMAX codec for decoding, and when the decoded raw audio data is available, it is returned by the read() method.

Similarly, the MPEG4Source object will block in the read() method until it has read one packet from the source file, which is then returned to the caller. With this design, elements can be more or less arbitrarily connected, as long as one element can handle the data type that the input element produces.

For pipelines, Stagefright contains these kinds of element implementations:

Other components

In Honeycomb/Ice Cream Sandwich, Stagefright has evolved further, with a second mode of operation in addition to the plain pipeline based system. Now, the ACodec class encapsulates an OpenMAX codec with a different kind of abstraction. This class isn’t to be used as a strict pipeline element, thus, it doesn’t implement the MediaSource interface. Instead, messages are passed to ACodec (which implements AHandler, for receiving messages) with buffers to pass to the OpenMAX codec. This is more similar to the behavior model of the actual OpenMAX codecs themselves.

In addition to the elements, Stagefright contains higher level classes such as AwesomePlayer that coordinates playback of a video file, other utility classes such as color converters, wrapping of OpenMAX codecs (providing a Binder C++ interface around OpenMAX, giving it the possibility to use the codecs from a different process even if the codecs themselves run within the media server), and a number of software implementations of codecs.

images/MediaFrameworkComponents.svg
An illustration of the general pipeline structure.
Note FileSource, MPEG4Source and OMXCodec all implement the MediaSource interface. MPEG4Extractor, OMXCodec, AudioPlayer, MediaWriter all use the MediaSource interface for reading their input from the preceding element, of which they don’t need to know anything else than what’s in MediaSource.
images/MediaFrameworkPipeline.svg
An example of a pipeline decoding audio and video from an MP4 file.
images/MediaFrameworkArchitecture.svg
A rough visualization of the split between the application process and media server.
images/MediaFrameworkIOMX.svg
Details of the IOMX interface.

      OpenMAX IL 5.1 Overview

OpenMAX IL in Stagefright

Stagefright provides wrapping around OMX core implementations. The main client api is called IOMX (frameworks/base/include/media/IOMX.h), which is a Binder interface with a proxy (BpOMX) and backend (BnOMX), allowing the caller and implementation to reside in separate processes. BnOMX is implemented by the OMX class in frameworks/base/media/libstagefright/omx/OMX.cpp, which uses the OMXMaster class in frameworks/base/media/libstagefright/omx/OMXMaster.cpp for querying and instantiating components from a number of different implementation sources. OMXMaster loads one or more plugins (OMXPluginBase, defined in frameworks/base/include/media/libstagefright/OMXPluginBase.h) that are analogous to normal OMX IL cores.

OMXMaster loads a shared library named libstagefrighthw.so, which should have a function named ZN7android15createOMXPluginEv(android::createOMXPlugin()) or createOMXPlugin (since ICS, both are tried, earlier, only the former was tried), that returns an OMXPluginBase pointer when called.

This library, libstagefrighthw.so, is vendor specific. Examples of implementations of this library, wrapping a normal OMX IL core, are available in:

All of these are very similar they simply load an OMX core from a shared library, load the OMX core function pointers from the library, and map the OMXPluginBase methods to the OMX core.

For software codecs, there’s no full proper OMX IL core in ICS, all decoders are mapped straight from the OMXPluginBase, implemented in frameworks/base/media/libstagefright/omx/SoftOMXPlugin.cpp. All software encoders (and in gingerbread, all software decoders too) are hooked up directly from the OMXCodec class, where the direct constructors of the software encoder classes are called, without any indirection via OMX like interfaces these software codecs implement the MediaSource interface directly. Due to this, these codecs cannot be accessed via the IOMX layer.

The development seems to be moving towards the OMX interfaces namely, in ICS, the software decoders have been converted to use OMX. The new ACodec class in ICS only uses codecs via the OMX interfaces.

      Adding a custom OpenMAX IL plugin

Implement an OMX core for the codecs

Examples:

The OMX core acts as a registry for the available codecs. If the build configuration already contains a vendor specific OMX core, the new OMX components can either be added to this OMX core, or a new separate OMX core can be added.

Build libstagefrighthw

Build libstagefright, containing an OMXPluginBase implementation that loads the OMX core. Examples:

If one chooses to add a second OMX core, a second libstagefrighthw (with a slightly different name) has to be created, and this also requires modifications to frameworks/base/media/libstagefright/omx/OMXMaster.cpp, in order to be able to handle two different vendor libraries at the same time. Therefore, it’s probably best to integrate new components within the current vendor OMX core if one already exists.

Implement OMX components for the codecs

This is the actual wrapping of the codecs. Simple examples (which don’t use the proper real OMX API but only stagefright’s own internal API) are available in

  1. frameworks/base/media/libstagefright/omx/SoftOMXComponent.cpp

  2. frameworks/base/media/libstagefright/omx/SimpleSoftOMXComponent.cpp

  3. frameworks/base/media/libstagefright/codecs/aacdec/SoftAAC.cpp

These contain all the logic any OMX component needs to have, but the external API isn’t the proper public OMX version, but only stagefright-internal ones. Real implementations with the proper public OMX API are available in e.g.:

Register the new OMX components

Register the new OMX components in frameworks/base/media/libstagefright/OMXCodec.cpp. Simply add the component names and the mime types it handles in the kDecoderInfo/kEncoderInfo tables. (OMXCodec in stagefright doesn’t query for which components implement certain roles, but blindly checks for all the components listed to support a certain mime type, and uses the first one that actually exists.)

   Telephony on Android

In this section, we’ll explore the inter-workings of the telephony stack. We’ll start with the standard Android Phone app, and trace the execution of placing a call all the way down to RIL daemon.

      Telephony Manager

Android Framework provides the Telephony Manager class in android.telephony package. The Telephony Manager allows you to monitor the state of the mobile network connection. However, Telephony Manager does not allow you to place or manage any calls. Only the Phone app can initiate and answer phone calls.

      Phone App

Note Some of this content comes from the actual AOSP source code, licensed under Apache 2 License.
images/TelephonyArchitecture.svg
Android Telephony Architecture Diagram
Overview

Phone app does come with some UI components, such as the dialer, but the most significant part is its handling of CALL and CALL_PRIVILEGED intents to do the actual dialing of a number.

OutgoingCallBroadcaster

OutgoingCallBroadcaster activity receives CALL and CALL_PRIVILEGED Intents, and broadcasts the ACTION_NEW_OUTGOING_CALL intent which allows other applications to monitor, redirect, or prevent the outgoing call. After the other applications have had a chance to see the ACTION_NEW_OUTGOING_CALL intent, it finally reaches the OutgoingCallReceiver, which passes the (possibly modified) intent on to the SipCallOptionHandler, which will ultimately start the call using the CallController.placeCall() API.

CallController

CallController handler is the phone app module in charge of call control. This is a singleton object which acts as the interface to the telephony layer (and other parts of the Android framework) for all user-initiated telephony functionality, like making outgoing calls.

This functionality includes things like:

The single CallController instance stays around forever; it’s not tied to the lifecycle of any particular Activity (like the InCallScreen). There’s also no implementation of onscreen UI here (that’s all in InCallScreen).

Note that this class does not handle asynchronous events from the telephony layer, like reacting to an incoming call; see CallNotifier for that. This class purely handles actions initiated by the user, like outgoing calls.

placeCall(Intent intent) initiates an outgoing call.

Here’s the most typical outgoing call sequence:

  1. OutgoingCallBroadcaster receives a CALL intent and sends the NEW_OUTGOING_CALL broadcast.

  2. The broadcast finally reaches OutgoingCallReceiver, which stashes away a copy of the original CALL intent and launches SipCallOptionHandler.

  3. SipCallOptionHandler decides whether this is a PSTN or SIP call (and in some cases brings up a dialog to let the user choose), and ultimately calls CallController.placeCall() (from the setResultAndFinish() method) with the stashed-away intent from step (2) as the "intent" parameter.

  4. Here in CallController.placeCall() we read the phone number or SIP address out of the intent and actually initiate the call, and simultaneously launch the InCallScreen to display the in-call UI.

  5. We handle various errors by directing the InCallScreen to display error messages or dialogs (via the InCallUiState "pending call status code" flag), and in some cases we also sometimes continue working in the background to resolve the problem (like in the case of an emergency call while in airplane mode). Any time that some onscreen indication to the user needs to change, we update the "status dialog" info in the inCallUiState and (re)launch the InCallScreen to make sure it’s visible.

PhoneUtils

PhoneUtils.placeCall(Context context, Phone phone, String number, Uri contactRef, boolean isEmergencyCall, Uri gatewayUri) dials the number using the phone passed in.

CallManager

CallManager, defined in PhoneApp class.

CallManager class provides an abstract layer for PhoneApp to access and control calls. It implements Phone interface.

CallManager provides call and connection control as well as channel capability.

There are three categories of APIs CallManager provided

  1. Call control and operation, such as dial() and hangup()

  2. Channel capabilities, such as CanConference()

  3. Register notification

public Connection dial(Phone phone, String dialString) in CallManager initiate a new voice connection. This happens asynchronously, so you cannot assume the audio path is connected (or a call index has been assigned) until PhoneStateChanged notification has occurred.

dial() uses PhoneProxy to implement Phone interface to make the call.

Phone Interface

Phone interface specifies the capabilities of the underlying phone system, weather GSM, CDMA, or SIP. We’ll assume GMS from here on.

GSMPhone

GSMPhone extends PhoneBase which in turn implements the Phone interface.

It uses GSMCallTracker to dial via CommandsInterface. This interface is implemented by RIL class.

RIL in turn uses RILRequest to send the requests to rild daemon.

Tip You can use adb logcat -b radio to see the radio-specific log messages.
RIL (Java)

RIL class is the implementation of the CommandsInterface that GSMPhone uses to dial out. This Java implementation writes out the commands to RIL daemon, the native code.

RIL uses RILSender and RILReceiver to send and receive messages from rild - the RIL Daemon. Unlike most of the other Android system services, RIL uses sockets for this inter-process communication, and not the Binder.

RIL Daemon

The RIL consists of two primary components:

Note Some of the following documentation comes from once available Android Platform Development Kit: Radio Layer Interface documentation. This documentation is no longer readily available from Google (deemed outdated) but is licensed under Apache 2.0 license. We’ve updated the references as originally provided.
RIL Initialization

Android initializes the telephony stack and the Vendor RIL at startup as described in the sequence below:

  1. RIL daemon reads rild.lib path and rild.libargs system properties to determine the Vendor RIL library to use and any initialization arguments to provide to the Vendor RIL.

  2. RIL daemon loads the Vendor RIL library and calls RIL_Init to initialize the RIL and obtain a reference to RIL functions.

  3. RIL daemon calls RIL_register on the Android telephony stack, providing a reference to the Vendor RIL functions

Details of this implementation are available in AOSP/hardware/ril/rild/rild.c.

RIL Interaction

There are two forms of communication that the RIL handles:

Solicited RIL Commands

The following snippet illustrates the interface for solicited commands:

images/RIL-solicited-commands.png
Example of solicited RIL commands

There are over sixty solicited commands grouped by the following families:

Unsolicited RIL Commands

The following snippet illustrates the interface for unsolicited commands:

images/RIL-unsolicited-commands.png
Example of unsolicited RIL commands

There are over ten unsolicited commands grouped by the following families:

Implementing the RIL

To implement a radio-specific RIL, create a shared library that implements a set of functions required by Android to process radio requests. The required functions are defined in the RIL header (AOSP/hardware/ril/include/telephony/ril.h).

The Android radio interface is radio-agnostic and the Vendor RIL can use any protocol to communicate with the radio. Android provides a reference Vendor RIL, using the Hayes AT command set, that you can use as a quick start for telephony testing and a guide for commercial vendor RILs. The source code for the reference RIL is found at AOSP/hardware/ril/reference-ril.

Compile your Vendor RIL as a shared library using the convention libril-<companyname>-<RIL version>.so, for example, libril-acme-124.so, where:

For reference implementation of RIL, see AOSP/hardware/ril/libril.

      Resources

   Device Policy on Android

images/DevicePolicyServiceArchitecture.svg
Device Policy Service Architecture

   Display on Android

images/SurfaceArchitecture.svg
Surface
images/SurfaceFlingerArchitecture.svg
Surface Flinger

      Resources

   Camera on Android

See Exposing the Android Camera Stack by Balwinder Kaur and Joe Rickson from Aptina Imaging, Inc.

      Resources

   NFC on Android

images/NfcServiceArchitecture.svg
NFC Service Architecture
Note This service is deployed as an application, /system/app/Nfc.apk!

   Review Questions

  1. Name at least five application framework services?

  2. What is the most typical communication channel for a client to communicate with its service?

  3. What’s the difference between managers and services?

  4. How do we get the list of all application framework services?

  5. What is the purpose of the servicemanager process?

  6. How do clients/services get access to the servicemanager process?

  7. What role does JNI play in most application framework services?

  8. What is the life-cycle of most application framework services?

  9. What is the relationship between application framework services and daemons?

  10. How do most services communicate with their daemons?

  11. What is the purpose of installd and who uses it and when?

  12. What is the purpose of wpa_supplicant and who uses it?

  13. Unlike the Vibrator (or Power) service, the Alarm service does not have what?

  14. How do services (like Location service) provide call-backs to their clients?

  15. What is AudioFlinger and what is its purpose?

  16. What is the role of Audio Policy Service?

  17. What is Stagefright and what is its purpose?

  18. How do hardware manufactures take advantage of Stagefright?

  19. How does the telephony stack differ from most other services?

  20. What is SurfaceFlinger and what is its purpose?

  21. How does NFC Service differ from most other services?

Customizing Android

Note The following steps assume building Android JB 4.1.1 for ARMv7 on Ubuntu 10.04 x86_64 or MacOS X 10.7 and that the Android sources are available under a directory pointed to by $AOSP_HOME environment variable. (e.g. we could set AOSP_HOME=~/aosp/).
Note The complete code (which we’ll develop in this section) for Marakana Alpha is available at https://github.com/marakana/alpha
The code for Marakana Alpha SDK add-on is available https://github.com/marakana/alpha_sdk_addon

   Setting up Custom Device Directory Structure

While we could overlay our device’s custom components over the existing AOSP source tree, that makes it harder to deal with future OS upgrades. Instead, we will create a self-contained directory structure to host our device.

  1. Go to AOSP directory 

    $ cd $AOSP_HOME
    Note We assume that you set AOSP_HOME to the root of your Android Open Source Project source directory (e.g. ~/aosp/). Unless otherwise stated, the rest of the file paths are assumed to be relative to this directory.

  2. Create our vendor (e.g. mararkana) directory 

    $ mkdir device/marakana/

  3. Now create our device (e.g. alpha) sub-directory: 

    $ mkdir device/marakana/alpha

   Registering our Device with Android’s Build System

We now want to add our device to the Android’s lunch list

Note Remember that $ source build/envsetup.sh registers lunch combos that we can later build

  1. Create vendorsetup.sh file for our device (the name of this file is fixed):

    device/marakana/alpha/vendorsetup.sh
    add_lunch_combo full_marakana_alpha-eng
add_lunch_combo full_marakana_alpha-userdebug
add_lunch_combo full_marakana_alpha-user

  2. Re-build the lunch list: 

    $ source build/envsetup.sh
including device/asus/grouper/vendorsetup.sh
including device/generic/armv7-a-neon/vendorsetup.sh
including device/generic/armv7-a/vendorsetup.sh
including device/marakana/alpha/vendorsetup.sh
including device/moto/wingray/vendorsetup.sh
including device/samsung/crespo/vendorsetup.sh
including device/samsung/maguro/vendorsetup.sh
including device/ti/panda/vendorsetup.sh
including sdk/bash_completion/adb.bash

  3. Finally, we can check to see that our device now appear in the lunch menu: 

    $ lunch

You're building on Linux

Lunch menu... pick a combo:
     1. full-eng
     2. full_x86-eng
     3. vbox_x86-eng
     4. full_grouper-userdebug
     5. mini_armv7a_neon-userdebug
     6. mini_armv7a-userdebug
     7. full_marakana_alpha-eng
     8. full_marakana_alpha-userdebug
     9. full_marakana_alpha-user
     10. full_wingray-userdebug
     11. full_crespo-userdebug
     12. full_maguro-userdebug
     13. full_maguro-eng
     14. full_panda-userdebug

Which would you like? [full-eng]
    • We are not yet ready to select it here, because we have not yet provided the necessary makefiles for full_marakana_alpha. If we do, we’ll get: 
      $ lunch full_marakana_alpha-eng
build/core/product_config.mk:203: *** No matches for product "full_marakana_alpha".  Stop.

** Don't have a product spec for: 'full_marakana_alpha'
** Do you have the right repo manifest?

   Adding the Makefile Plumbing for our Device

We now need to add basic support for building our device.

  1. Start by creating a our own AndroidProducts.mk file, which simply defines the actual makefiles to be used when building our device:

    device/marakana/alpha/AndroidProducts.mk
    PRODUCT_MAKEFILES := $(LOCAL_DIR)/full_alpha.mk
    Note The only purpose of AndroidProducts.mk file (whose name is fixed) is to set PRODUCT_MAKEFILES to a list of product makefiles to expose to the build system. The only external variable it can use is LOCAL_DIR, whose value will be automatically set to the directory containing this file.

  2. Before we create the full_alpha.mk makefile, let’s create a common.mk where we’ll configure settings that will be shared between our Alpha device and our Alpha SDK Addon:

    device/marakana/alpha/common-v1.mk
    # Since this file can also be referenced by alpha-sdk_addon
# we cannot assume LOCAL_PATH points to the directory where
# this file is located. Instead, we create another variable
# to capture this directory.
MY_PATH := $(LOCAL_PATH)/../alpha

# Include all makefiles in sub-directories (one level deep)
include $(call all-subdir-makefiles)

  3. Now we are ready to create the main build-file for our device (we call it full_alpha following the example from device/samsung/crespo/full_crespo.mk):

    device/marakana/alpha/full_alpha.mk
    # Defines a list of languages to be supported by our device
$(call inherit-product, $(SRC_TARGET_DIR)/product/languages_small.mk)

# Defines the rules for building the base Android platform,
# but itself is not specialized for any particular device
$(call inherit-product, $(SRC_TARGET_DIR)/product/generic.mk)

# Discard inherited values and use our own instead.
PRODUCT_NAME := full_marakana_alpha
PRODUCT_DEVICE := alpha
PRODUCT_MODEL := Full Marakana Alpha Image for Emulator

# Include the common definitions and packages
include $(LOCAL_PATH)/common.mk
    About Product Inheritance

    Our device inherits from the generic product:

    $(call inherit-product, $(SRC_TARGET_DIR)/product/generic.mk)

    which resolves to build/target/product/generic.mk file.

    This generic product (file) in turn inherits from:

    • build/target/product/generic_no_telephony.mk
      • frameworks/base/data/fonts/fonts.mk
      • external/lohit-fonts/fonts.mk
      • frameworks/base/data/keyboards/keyboards.mk
      • build/target/product/core.mk
    • build/target/product/telephony.mk

    The combination of these files set up compilation rules that define which modules (a.k.a. PRODUCT_PACKAGES) get included in the final product (i.e. ROM).

    Additionally, Android’s build system also defines GRANDFATHERED_USER_MODULES, which get included into every product, and are defined by build/core/user_tags.mk.

  4. Next, we’ll import some boiler-plate make/support files from the "generic" board - since our device will run on the emulator: 

    $ cp build/target/board/generic/BoardConfig.mk device/marakana/alpha/.
$ cp build/target/board/generic/AndroidBoard.mk device/marakana/alpha/.
$ cp build/target/board/generic/device.mk device/marakana/alpha/.
$ cp build/target/board/generic/system.prop device/marakana/alpha/.
    Tip For x86, copy from build/target/board/generic_x86/ instead.
    • AndroidBoard.mk - is essentially empty
    • device.mk - sets up basic emulator-specific configuration files
    • system.prop - used to set system-wide properties (here, just RIL settings for the emulator)
    • BoardConfig.mk - defines our device board’s kernel/hardware capabilities:
      device/marakana/alpha/BoardConfig.mk
      # config.mk
#
# Product-specific compile-time definitions.
#

# The generic product target doesn't have any hardware-specific pieces.
TARGET_NO_BOOTLOADER := true
TARGET_NO_KERNEL := true

# Note: we build the platform images for ARMv7-A _without_ NEON.
#
# Technically, the emulator supports ARMv7-A _and_ NEON instructions, but
# emulated NEON code paths typically ends up 2x slower than the normal C code
# it is supposed to replace (unlike on real devices where it is 2x to 3x
# faster).
#
# What this means is that the platform image will not use NEON code paths
# that are slower to emulate. On the other hand, it is possible to emulate
# application code generated with the NDK that uses NEON in the emulator.
#
TARGET_ARCH_VARIANT := armv7-a
TARGET_CPU_ABI := armeabi-v7a
TARGET_CPU_ABI2 := armeabi

HAVE_HTC_AUDIO_DRIVER := true
BOARD_USES_GENERIC_AUDIO := true

# no hardware camera
USE_CAMERA_STUB := true

# Set /system/bin/sh to ash, not mksh, to make sure we can switch back.
TARGET_SHELL := ash

# Enable dex-preoptimization to speed up the first boot sequence
# of an SDK AVD. Note that this operation only works on Linux for now
ifeq ($(HOST_OS),linux)
WITH_DEXPREOPT := true
endif

# Build OpenGLES emulation guest and host libraries
BUILD_EMULATOR_OPENGL := true

# Build and enable the OpenGL ES View renderer. When running on the emulator,
# the GLES renderer disables itself if host GL acceleration isn't available.
USE_OPENGL_RENDERER := true
      About CPU Architecture

      Because we copied build/target/board/generic/BoardConfig.mk we inherited the following:

      TARGET_ARCH_VARIANT := armv7-a
TARGET_CPU_ABI := armeabi-v7a
TARGET_CPU_ABI2 := armeabi

      Alternatively, we could have built our image for x86, either by copying build/target/board/generic_x86/BoardConfig.mk or by changing our own device/marakana/alpha/BoardConfig.mk to say the following:

      TARGET_ARCH := x86
TARGET_ARCH_VARIANT := x86-atom
TARGET_CPU_ABI := x86

  5. Finally, let’s change the default wallpaper for our device (for fun) by taking advantage of the build system’s overlay feature:

    Note

    The overlay feature enables us to override specific files in the Android framework/app res/ folders, without having to change anything in the AOSP source directories. This is usually used to customize the look and feel as well as app/system-specific configuration settings.

    In our case, we want to replace frameworks/base/core/res/res/drawable-nodpi/default_wallpaper.jpg file in the final image because that’s what the Launcher2 defaults to.

    1. Make the specific overlay/ directory: 

      $ mkdir -p device/marakana/alpha/overlay/frameworks/base/core/res/res/drawable-nodpi

    2. Push our default_wallpaper.jpg to it:

      device/marakana/alpha/overlay/frameworks/base/core/res/res/drawable-nodpi/default_wallpaper.jpg
      device/marakana/alpha/overlay/frameworks/base/core/res/res/drawable-nodpi/default_wallpaper.jpg:
      Note This image is also available on GitHub

    3. Finally, we need to tell the build-system to use our overlay/ directory:

      device/marakana/alpha/common.mk:
      …
# Enable overlays
DEVICE_PACKAGE_OVERLAYS := $(MY_PATH)/overlay

   Generating Our Own Platform Signing Keys (Recommended)

Before we compile our device, we should generate our own platform signing keys [Platform_Keys], otherwise our device will not pass CTS.

  1. Define our subject/issuer info: 

    $ SIGNER="/C=US/ST=California/L=San Francisco/O=Marakana Inc./OU=Android/CN=Android Platform Signer/emailAddress=android@marakana.com"

  2. Remove the existing keys (it does not hurt to back them up first!): 

    $ rm build/target/product/security/*.p*

  3. Generate the platform key: 

    $ echo | development/tools/make_key build/target/product/security/platform "$SIGNER"
creating build/target/product/security/platform.pk8 with no password
Generating RSA private key, 2048 bit long modulus
....................+++
..........................................................+++
e is 3 (0x3)

  4. Generate the shared key: 

    $ echo | development/tools/make_key build/target/product/security/shared "$SIGNER"
creating build/target/product/security/shared.pk8 with no password
Generating RSA private key, 2048 bit long modulus
..................................................................................................+++
............+++
e is 3 (0x3)

  5. Generate the media key: 

    $ echo | development/tools/make_key build/target/product/security/media "$SIGNER"
creating build/target/product/security/media.pk8 with no password
Generating RSA private key, 2048 bit long modulus
...................+++
....................+++
e is 3 (0x3)

  6. Generate the testkey key: 

    $ echo | development/tools/make_key build/target/product/security/testkey "$SIGNER"
creating build/target/product/security/testkey.pk8 with no password
Generating RSA private key, 2048 bit long modulus
....................+++
................................................+++
e is 3 (0x3)

  7. Verify that our keys have been created: 

    $ ls -1 build/target/product/security/*.p*
build/target/product/security/media.pk8
build/target/product/security/media.x509.pem
build/target/product/security/platform.pk8
build/target/product/security/platform.x509.pem
build/target/product/security/shared.pk8
build/target/product/security/shared.x509.pem
build/target/product/security/testkey.pk8
build/target/product/security/testkey.x509.pem

  8. Check that our specific subject/issuer has been used: 

    $ openssl x509 -noout -subject -issuer -in  build/target/product/security/platform.x509.pem
subject= /C=US/ST=California/L=San Francisco/O=Marakana Inc./OU=Android/CN=Android Platform Signer/emailAddress=android@marakana.com
issuer= /C=US/ST=California/L=San Francisco/O=Marakana Inc./OU=Android/CN=Android Platform Signer/emailAddress=android@marakana.com
    Caution The build/target/product/security/.pk8 files are the private keys (.x509.pem are the certificates), and we need to make sure to keep them safe and secure - especially since we did not encrypt them!

   Building our Device For The First Time

We are now ready to try out our "custom" device - though there is nothing truly custom yet, except for the PRODUCT_* settings and our own product keys.

  1. For good measure, re-register our device: 

    $ source build/envsetup.sh
including device/marakana/alpha/vendorsetup.sh
…

  2. Now we can lunch of our device: 

    $ lunch full_marakana_alpha-eng

============================================
PLATFORM_VERSION_CODENAME=REL
PLATFORM_VERSION=4.0.3
TARGET_PRODUCT=full_marakana_alpha
TARGET_BUILD_VARIANT=eng
TARGET_BUILD_TYPE=release
TARGET_BUILD_APPS=
TARGET_ARCH=arm
TARGET_ARCH_VARIANT=armv7-a
HOST_ARCH=x86
HOST_OS=linux
HOST_BUILD_TYPE=release
BUILD_ID=IML74K
============================================

  3. We can now compile our device: 

    $ make -j10
…
Installed file list: out/target/product/alpha/installed-files.txt
Target system fs image: out/target/product/alpha/obj/PACKAGING/systemimage_intermediates/system.img
Install system fs image: out/target/product/alpha/system.img
    Tip Adjust the value of -j based on your system’s available cores, memory, and disk I/O.

  4. Finally, we can run it 

    $ $ANDROID_HOST_OUT/bin/emulator &

  5. And we should see

    screens/MarakanaAlpha-Wallpaper.png
    Custom Wallpaper
    screens/MarakanaAlpha-AboutPhone-v1.png
    Custom Model Number on the About Screen

   Adding a Custom Kernel to our Device

Our device would work fine with the provided QEMU-based (i.e. emulator-specific) kernel, but in this section, we will configure a custom kernel ([Android_Building_Linux_Kernel]) so that we can:

About QEMU Kernels

Pre-build QEMU kernels are available in AOSP source tree:

  • Android 4.0 (ICS)
    • prebuilt/android-arm/kernel/kernel-qemu-armv7 for ARMv7
    • prebuilt/android-arm/kernel/kernel-qemu for ARMv5
    • prebuilt/android-x86/kernel/kernel-qemu for x86
  • Android 4.1 (JB)
    • prebuilts/qemu-kernel/arm/kernel-qemu-armv7 for ARMv7
    • prebuilts/qemu-kernel/arm/kernel-qemu for ARMv5
    • prebuilts/qemu-kernel/x86/kernel-qemu for x86

The steps outlined here are similar to [Android_Building_Linux_Kernel_for_Emulator]:

  1. Create a directory to host our kernel sources: 

    $ mkdir ~/kernel/
$ cd ~/kernel/

  2. Clone the Goldfish kernel sources: 

    $ git clone https://android.googlesource.com/kernel/goldfish.git
$ cd goldfish/

  3. Check out android-goldfish-2.6.29 branch of the kernel we wish to build: 

    $ git checkout -t remotes/origin/android-goldfish-2.6.29

  4. Set the architecture variable to match TARGET_ARCH_VARIANT we set in device/marakana/alpha/BoardConfig.mk: 

    $ export ARCH=arm
    Tip

    When compiling for x86, set the following instead:

    $ export ARCH=x86

  5. Create the default kernel configuration file (.config): 

    $ make goldfish_armv7_defconfig
    Tip

    When compiling for ARMv5 or x86, run the following instead:

    $ make goldfish_defconfig

  6. Change the Hardware name from "Goldfish" to "Marakana Alpha Board":

    arch/arm/mach-goldfish/board-goldfish.c:
    …
MACHINE_START(GOLDFISH, "Marakana Alpha Board")
…
    About the Hardware (Board) Name

    As we already know, when the Android kernel loads, it runs init, which configures itself from init.rc and init.hardware_.rc configuration files.

    The hardware name is extracted from /proc/cpuinfo:

    $ adb shell cat /proc/cpuinfo |grep Hardware
Hardware    : Goldfish

    Basically, system/core/init/init.c:main() uses system/core/init/util.c:get_hardware_name(…) to parse /proc/cpuinfo and extract the hardware name. This name is converted to lower-case and all the space characters are trimmed.

    Android’s ueventd process (started by init) also configures itself in a similar way, by loading ueventd.rc and ueventd.hardware.rc configuration files.

    Since in our case we changed the name of our hardware to "Marakana Alpha Board", we will also need to create the corresponding: init.marakanaalphaboard.rc and ueventd.marakanaalphaboard.rc files.

    Alternatively, the hardware name may also be set by the bootloader via kernel boot options. These are exposed to the userspace via /proc/cmdline (e.g. … androidboot.hardware=goldfish …)

  7. Configure the kernel 

    $ make menuconfig

    1. Set General setup → Local version to -marakana-alpha-release

    2. Select Enable loadable module support and within it also select Module unloading and within that Forced module unloading

    3. Customize the rest as desired

  8. Set the cross compiler: 

    $ export CROSS_COMPILE=$ANDROID_EABI_TOOLCHAIN/arm-linux-androideabi-
    Tip

    When compiling for x86, set the following instead:

    $ export REAL_CROSS_COMPILE=$ANDROID_EABI_TOOLCHAIN/i686-android-linux-
$ export CROSS_COMPILE=$AOSP/external/qemu/distrib/kernel-toolchain/android-kernel-toolchain-

    Android’s x86 compiler is NDK-compatible, so it enforces -mfpmath=sse and -fpic by default. When building the kernel, we need to disable this.

    In this case, $CROSS_COMPILE points to a toolchain of shell scripts, which add -mfpmath=387 -fno-pic flags before calling into $REAL_CROSS_COMPILE toolchain.
    About Cross Compilers

    The environment variable ANDROID_EABI_TOOLCHAIN was initialized by lunch when we ran lunch full_marakana_alpha-eng.

    Its value varies based on the version of Android, host architecture, and target architecture:

    • Android 4.0 (ICS)
      • MacOS X
        • arm: $AOSP_HOME/prebuilt/darwin-x86/toolchain/arm-linux-androideabi-4.4.x/bin
        • x86: $AOSP_HOME/prebuilt/darwin-x86/toolchain/arm-linux-androideabi-4.4.x/bin
      • Linux
        • arm: $AOSP_HOME/prebuilt/linux-x86/toolchain/arm-linux-androideabi-4.4.x/bin
        • x86: $AOSP_HOME/prebuilt/linux-x86/toolchain/arm-linux-androideabi-4.4.x/bin
    • Android 4.1 (JB)
      • MacOS X
        • arm: $AOSP_HOME/prebuilts/gcc/darwin-x86/arm/arm-linux-androideabi-4.6/bin
        • x86: $AOSP_HOME/prebuilts/gcc/darwin-x86/arm/arm-linux-androideabi-4.6/bin
      • Linux
        • arm: $AOSP_HOME/prebuilts/gcc/linux-x86/x86/i686-android-linux-4.4.3/bin
        • x86: $AOSP_HOME/prebuilts/gcc/linux-x86/arm/arm-linux-androideabi-4.6/bin

  9. Compile the kernel 

    $ make -j4
    Warning We’ve had issues compiling the kernel on Mac OS X Lion (10.7).
    About The Build Kernel Script

    AOSP comes with a convenient script to compile the Linux kernel for QEMU-based emulator: $AOSP_HOME/external/qemu/distrib/build-kernel.sh.

    We could use this script instead of setting up our own ARCH and CROSS_COMPILE options and running make.

    For example, to compile for x86, we could do:

    $ $AOSP_HOME/external/qemu/distrib/build-kernel.sh --arch=x86

    The target kernel will go to /tmp/kernel-qemu/ by default.

  10. Copy the compiled kernel to our device’s alpha/ directory: 

    $ cp arch/arm/boot/zImage $AOSP_HOME/device/marakana/alpha/kernel
    Tip For x86, copy arch/x86/boot/bzImage instead.
    Note In case you were not able to compile the kernel, you could also download a pre-built one (for ARMv7) from https://github.com/marakana/alpha/raw/master/kernel

  11. Now we can go back to the AOSP source 

    $ cd $AOSP_HOME/

  12. Enable our custom kernel in BoardConfig.mk (double-negation, nice!):

    device/marakana/alpha/BoardConfig.mk
    …
TARGET_NO_KERNEL := false
…
    Note This will cause the build system to rebuild the boot.img file, but because we are still working with the emulator, we’ll have to explicitly reference our kernel on startup.

  13. Now we are ready to create our own init and ueventd configuration files by copying the ones from Goldfish: 

    $ cp system/core/rootdir/etc/init.goldfish.rc device/marakana/alpha/init.marakanaalphaboard.rc
$ cp system/core/rootdir/etc/ueventd.goldfish.rc device/marakana/alpha/ueventd.marakanaalphaboard.rc
    Note Remember, marakanaalphaboard is our device’s new hardware name (lowercased and stripped of spaces).

  14. Next, we need to enable our kernel and copy the init and ueventd configuration files:

    device/marakana/alpha/common.mk
    …
# Enable our custom kernel
LOCAL_KERNEL := $(MY_PATH)/kernel
PRODUCT_COPY_FILES += $(LOCAL_KERNEL):kernel

# Copy our init and ueventd configuration files to the root
# file system (ramdisk.img -> boot.img)
PRODUCT_COPY_FILES += $(MY_PATH)/init.marakanaalphaboard.rc:root/init.marakanaalphaboard.rc
PRODUCT_COPY_FILES += $(MY_PATH)/ueventd.marakanaalphaboard.rc:root/ueventd.marakanaalphaboard.rc

  15. Re-build our ROM, which, in this case just creates a new out/target/product/alpha/boot.img (that we won’t actually use): 

    $ make -j10
…

  16. Restart the emulator (with our new kernel) 

    $ $ANDROID_HOST_OUT/bin/emulator -kernel out/target/product/alpha/kernel &
    Note We have to run our emulator with -kernel, because otherwise it would ignore the kernel in our newly built boot.img and instead use the default one (prebuilt/android-arm/kernel/kernel-qemu-armv7). This applies just to the emulator.

  17. Test 

    $ adb shell cat /proc/version
Linux version 2.6.29-marakana-alpha-release-g46b05b2 (sasa@thermal) (gcc version 4.4.3 (GCC) ) #3 Tue Jan 3 13:25:44 PST 2012
    Note The path to adb was added to our PATH when we $ source build/envsetup.sh (on a Linux host, it comes from out/host/linux-x86/bin/adb)

  18. We could also take a look at the updated About screen:

    screens/MarakanaAlpha-AboutPhone-v2.png

   Adding a Custom Native Library and Executable to our Device

   Using our Native Library via a Custom Daemon

   Exposing our Native Library via Java (i.e. JNI)

   Consuming our a Custom Java/JNI→Native Library via a Custom App (Optional)

   Exposing our Custom Library via a Custom IPC/Binder Service

Rather than have 3rd party applications direct access to "our" driver (via JNI/HAL), we may be better off providing access to the driver’s functionality via a Binder (AIDL-described) service.

   Building a Custom App Using a Custom Service Manager

   Creating a Custom SDK Add-on (Optional)

      Distributing our Custom SDK Add-on (Optional)

  1. First, we need to create a repository.xml file to describe our add-on and publish it to our server:

    https://marakana.com/external/android/sdk-addon/repository.xml
    <?xml version="1.0" encoding="UTF-8"?>

<sdk:sdk-addon
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xmlns:sdk="http://schemas.android.com/sdk/android/addon/1">

    <sdk:add-on>
        <sdk:name>Alpha Add-On</sdk:name>
        <sdk:api-level>15</sdk:api-level>
        <sdk:vendor>Marakana</sdk:vendor>
        <sdk:revision>1</sdk:revision>
        <sdk:description>Android + Marakana Alpha Add-on, API 15, revision 1 </sdk:description>
        <sdk:desc-url>http://marakana.com/external/android/sdk-addon/</sdk:desc-url>
        <sdk:uses-license ref="marakana-android-addon-license" />
        <sdk:archives>
            <sdk:archive os="any">
                <sdk:size>96382546</sdk:size>
                <sdk:checksum type="sha1">5cec8cc3f3064441cb96a50e2b8aa528681ffc78</sdk:checksum>
                <sdk:url>marakana_alpha_sdk_addon_api-15_r1.zip</sdk:url>
            </sdk:archive>
        </sdk:archives>
        <sdk:libs>
        </sdk:libs>
    </sdk:add-on>

    <sdk:license type="text" id="marakana-android-addon-license">
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this SDK addon except in compliance with the License.
You may obtain a copy of the License at: http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
    </sdk:license>
</sdk:sdk-addon>
    Note We have to make sure that we adjust the info as necessary to reflect our particular site/addon.

  2. Next, we need to upload our addon to our site, relative to <sdk:sdk-addon> → <sdk:add-on> → <sdk:desc-url> URL (e.g. https://marakana.com/external/android/sdk-addon/marakana_alpha_sdk_addon_api-15_r1.zip)

  3. We are now ready to test it out in our Android SDK Manager

    1. Go to Tools → Manage Add-on Sites… in the menu bar

    2. Click on New…

    3. Enter the base URL in the URL field (e.g. https://marakana.com/external/android/sdk-addon/) and click on OK

    4. Click on Close to dismiss the Add-on Sites window

    5. Under Packages we should now see Alpha Add-On by Marakana, with a status Not installed

    6. Click on the checkbox next to our package

    7. Click on Install 1 package… button

    8. Accept the license terms and click on Install

  4. Test that it works by creating an AVD based on our add-on and/or using it in Eclipse

   Review Questions

  1. What is a home directory for a new custom board within AOSP source-tree?

  2. What is the purpose of vendorsetup.sh?

  3. What does $(call inherit-product, $(SRC_TARGET_DIR)/product/generic.mk) do?

  4. What is the significance of PRODUCT_PACKAGES?

  5. What is the purpose of BoardConfig.mk?

  6. What is the purpose of development/tools/make_key and why do we want to use it?

  7. What is the significance of /proc/cpuinfo?

  8. When/why do we use include $(call all-subdir-makefiles)?

  9. What is the purpose of LOCAL_MODULE?

  10. What is the purpose of LOCAL_MODULE_TAGS?

  11. How do we compile individual modules efficiently?

  12. Name at least three include $(BUILD_XXX) targets.

  13. What do we need to do in order to launch a custom daemon on startup?

  14. How do we expose custom Java system libraries to applications?

  15. How do applications consume custom Java system libraries?

  16. When do we use include $(BUILD_PREBUILT)?

  17. When/where do we use $(JNI_H_INCLUDE)?

  18. What is the purpose of {@hide}?

  19. When do we use include $(BUILD_DROIDDOC)?

  20. What do we need to do in order to register a custom service to servicemanager?

  21. When/why would we use LOCAL_JAVA_LIBRARIES += framework?

  22. When/why would we use LOCAL_CERTIFICATE := platform?

  23. What does include $(BUILD_PACKAGE) build?

  24. What is SDK add-on and when/why would we want to build/use one?

  25. How do we distribute custom SDK add-ons?

USB on Android

   Objectives of USB Module

   Overview of Android USB Support

      Android USB Host and Accessory Modes

Android supports a variety of USB peripherals and Android USB accessories (hardware that implements the Android accessory protocol) through two modes: USB host and USB accessory.

In USB host mode, the Android-powered device acts as the host.

In USB accessory mode, the external USB hardware act as the USB hosts.

      Android USB Support

USB accessory and host modes are directly supported in Android 3.1 (API level 12) or newer platforms.

Note Support for USB host and accessory modes are ultimately dependent on the device’s hardware, regardless of platform level. You can filter for devices that support USB host and accessory through a <uses-feature> element, as discussed later.

   Android USB Host Mode

      API Overview

The android.hardware.usb package contains the following classes supporting USB host mode:

UsbManager

Allows you to enumerate and communicate with connected USB devices.

UsbDevice

Represents a connected USB device and contains methods to access its identifying information, interfaces, and endpoints.

UsbInterface

Represents an interface of a USB device, which defines a set of functionality for the device. A device can have one or more interfaces on which to communicate on.

UsbEndpoint

Represents an interface endpoint, which is a communication channel for this interface. An interface can have one or more endpoints, and usually has input and output endpoints for two-way communication with the device.

UsbDeviceConnection

Represents a connection to the device, which transfers data on endpoints. This class allows you to send data back and forth sychronously or asynchronously.

UsbRequest

Represents an asynchronous request to communicate with a device through a UsbDeviceConnection.

UsbConstants

Defines USB constants that correspond to definitions in linux/usb/ch9.h of the Linux kernel.

In general, you:

  1. Obtain a UsbManager to retrieve the desired UsbDevice

  2. Find the appropriate UsbInterface and the UsbEndpoint of that interface to communicate on

  3. Open a UsbDeviceConnection to communicate with the USB device

In most situations, you need to use all of these classes (UsbRequest is only required if you are doing asynchronous communication) when communicating with a USB device.

      Using the Manifest to Require USB Support

Not all Android-powered devices are guaranteed to support the USB host APIs.

To indicate that your application requires USB host support, add the following elements to your application’s manifest:

      Working with Devices

When users connect USB devices to an Android-powered device, the Android system can determine whether your application is interested in the connected device. If so, you can set up communication with the device if desired.

To do this, your application has to:

  1. Discover connected USB devices either by:

    • Using an intent filter to be notified when the user connects a USB device, or
    • Enumerating USB devices that are already connected.

  2. Ask the user for permission to connect to the USB device, if not already obtained.

  3. Communicate with the USB device by reading and writing data on the appropriate interface endpoints.

      Discovering a Device Using an Intent Filter, The Manifest

To have your application discover a particular USB device, you can specify an intent filter to filter for the android.hardware.usb.action.USB_DEVICE_ATTACHED intent.

In your activity, you can obtain the UsbDevice that represents the attached device from the intent like this:

UsbDevice device = (UsbDevice) intent.getParcelableExtra(UsbManager.EXTRA_DEVICE);

The following example shows how to declare the intent filter in your application’s manifest:

<activity ...>
    ...
    <intent-filter>
        <action android:name="android.hardware.usb.action.USB_DEVICE_ATTACHED" />
    </intent-filter>

    <meta-data android:name="android.hardware.usb.action.USB_DEVICE_ATTACHED"
        android:resource="@xml/device_filter" />
</activity>

The <meta-data> element points to an external XML resource file that declares identifying information about the device that you want to detect.

      Discovering a Device Using an Intent Filter, The Resource File

In the XML resource file, declare <usb-device> elements for the USB devices that you want to filter.

Save the resource file in your application’s res/xml/ directory.

      Enumerating USB Devices

If your application is interested in inspecting all of the USB devices currently connected while your application is running, it can enumerate devices on the bus.

UsbManager manager = (UsbManager) getSystemService(Context.USB_SERVICE);
// ...
HashMap<String, UsbDevice> deviceList = manager.getDeviceList();
UsbDevice device = deviceList.get("deviceName");

If desired, you can also just obtain an iterator from the hash map and process each device one by one:

UsbManager manager = (UsbManager) getSystemService(Context.USB_SERVICE);
// ...
HashMap<String, UsbDevice> deviceList = manager.getDeviceList();
Iterator<UsbDevice> deviceIterator = deviceList.values().iterator();
while (deviceIterator.hasNext()) {
    UsbDevice device = deviceIterator.next()
    // ...
}

      Obtaining Permission to Communicate With a Device

Before communicating with the USB device, your application must have permission from your users.

Note If your application uses an intent filter to discover USB devices as they’re connected, it automatically receives permission if the user allows your application to handle the intent. If not, you must request permission explicitly in your application before connecting to the device.

Explicitly asking for permission might be necessary in some situations such as when your application enumerates USB devices that are already connected and then wants to communicate with one.

To explicitly obtain permission:

  1. Call UsbManager.requestPermission().

  2. The call to requestPermission() displays a dialog to the user asking for permission to connect to the device.

  3. The system generates a broadcast intent with a boolean EXTRA_PERMISSION_GRANTED extra indicating the user’s response.

The following sample code shows how to create a broadcast receiver to process the response:

private static final String ACTION_USB_PERMISSION = "com.android.example.USB_PERMISSION";

private final BroadcastReceiver mUsbReceiver = new BroadcastReceiver() {

    public void onReceive(Context context, Intent intent) {
        String action = intent.getAction();
        if (ACTION_USB_PERMISSION.equals(action)) {
            synchronized (this) {
                UsbDevice device = (UsbDevice)intent.getParcelableExtra(UsbManager.EXTRA_DEVICE);

                if (intent.getBooleanExtra(UsbManager.EXTRA_PERMISSION_GRANTED, false)) {
                    if (device != null) {
                      // Call method to set up device communication
                   }
                }
                else {
                    Log.d(TAG, "permission denied for device " + device);
                }
            }
        }
    }
};

To register the broadcast receiver, add this in your onCreate() method in your activity:

IntentFilter filter = new IntentFilter(ACTION_USB_PERMISSION);
registerReceiver(mUsbReceiver, filter);

To display the dialog that asks users for permission to connect to the device, call the UsbManager.requestPermission() method:

UsbManager mUsbManager = (UsbManager) getSystemService(Context.USB_SERVICE);
UsbDevice device;
// ...
mPermissionIntent = PendingIntent.getBroadcast(this, 0, new Intent(ACTION_USB_PERMISSION), 0);
mUsbManager.requestPermission(device, mPermissionIntent);

      Communicating With a Device

Communication with a USB device can be either synchronous or asynchronous.

To properly set up communication with a device, you need to obtain the appropriate UsbInterface and UsbEndpoint of the device that you want to communicate on and send requests on this endpoint with a UsbDeviceConnection. In general, your code should:

The following code snippet is a trivial way to do a synchronous data transfer. Your code should have more logic to correctly find the correct interface and endpoints to communicate on and also should do any transferring of data in a different thread than the main UI thread:

private byte[] bytes = "hello usb".getBytes();
private static int TIMEOUT = 0;
private boolean forceClaim = true;

...

UsbInterface intf = device.getInterface(0);
UsbEndpoint endpoint = intf.getEndpoint(0);
UsbDeviceConnection connection = mUsbManager.openDevice(device);
connection.claimInterface(intf, forceClaim);
connection.bulkTransfer(endpoint, bytes, bytes.length, TIMEOUT); // Do in another thread

To send data asynchronously, use the UsbRequest class to initialize and queue an asynchronous request, then wait for the result with requestWait().

      Terminating Communication With a Device

When you are done communicating with a device or if the device was detached, close the UsbInterface and UsbDeviceConnection by calling releaseInterface() and close().

To listen for detached events, create a broadcast receiver like below:

BroadcastReceiver mUsbReceiver = new BroadcastReceiver() {
    public void onReceive(Context context, Intent intent) {
        String action = intent.getAction();

      if (UsbManager.ACTION_USB_DEVICE_DETACHED.equals(action)) {
            UsbDevice device = (UsbDevice)intent.getParcelableExtra(UsbManager.EXTRA_DEVICE);
            if (device != null) {
                // call your method that cleans up and closes communication with the device
            }
        }
    }
};

Creating the broadcast receiver within the application, and not the manifest, allows your application to handle detached events only while it is running.

   Android USB Accessory Mode

      Choosing the Right USB Accessory APIs

The USB accessory APIs were introduced to the platform in Android 3.1. They are also available in Android 2.3.4 using the Google APIs add-on library.

      API Overview

The following classes support the USB accessory APIs:

UsbManager

Allows you to enumerate and communicate with connected USB accessories.

  • If you are using the add-on library, you must obtain the UsbManager object in the following manner: 
    UsbManager manager = UsbManager.getInstance(this);
  • If you are not using the add-on library, you must obtain the UsbManager object in the following manner: 
    UsbManager manager = (UsbManager) getSystemService(Context.USB_SERVICE);
UsbAccessory

Represents a USB accessory and contains methods to access its identifying information.

  • When you filter for a connected accessory with an intent filter, the UsbAccessory object is contained inside the intent that is passed to your application. If you are using the add-on library, you must obtain the UsbAccessory object in the following manner: 
    UsbAccessory accessory = UsbManager.getAccessory(intent);
  • If you are not using the add-on library, you must obtain the UsbAccessory object in the following manner: 
    UsbAccessory accessory = (UsbAccessory) intent.getParcelableExtra(UsbManager.EXTRA_ACCESSORY);

Because the add-on library is a wrapper for the framework APIs, the classes that support the USB accessory feature are similar. You can use the reference documentation for the android.hardware.usb even if you are using the add-on library.

      Android Manifest Requirements

Not all Android-powered devices are guaranteed to support the USB accessory APIs.

To indicate that your application requires USB accessory support, add the following elements to your application’s manifest:

      Working with Accessories

When users connect USB accessories to an Android-powered device, the Android system can determine whether your application is interested in the connected accessory. If so, you can set up communication with the accessory if desired.

To do this, your application has to:

  1. Discover connected accessories by using an intent filter that filters for accessory attached events or by enumerating connected accessories and finding the appropriate one.

  2. Ask the user for permission to communicate with the accessory, if not already obtained.

  3. Communicate with the accessory by reading and writing data on the appropriate interface endpoints.

      Discovering a Device Using an Intent Filter, The Manifest

To have your application discover a particular USB device, you can specify an intent filter to filter for the android.hardware.usb.action.USB_ACCESSORY_ATTACHED intent.

In your activity, you can obtain the UsbAccessory that represents the attached device from the intent.

The following example shows how to declare the intent filter in your application’s manifest:

<activity ...>
    ...
    <intent-filter>
        <action android:name="android.hardware.usb.action.USB_ACCESSORY_ATTACHED" />
    </intent-filter>

    <meta-data android:name="android.hardware.usb.action.USB_ACCESSORY_ATTACHED"
        android:resource="@xml/accessory_filter" />
</activity>

The <meta-data> element points to an external XML resource file that declares identifying information about the device that you want to detect.

      Discovering a Device Using an Intent Filter, The Resource File

In the XML resource file, declare <usb-accessory> elements for the accessories that you want to filter.

Save the resource file in your application’s res/xml/ directory.

      Enumerating Accessories

You can have your application enumerate accessories that have identified themselves while your application is running.

Use the UsbManager.getAccessoryList() method to get an array all the USB accessories that are connected:

UsbManager manager = (UsbManager) getSystemService(Context.USB_SERVICE);
UsbAccessory[] accessoryList = manager.getAcccessoryList();
Note Currently, only one connected accessory is supported at one time, but the API is designed to support multiple accessories in the future.

      Obtaining Permission to Communicate With an Accessory

Before communicating with the USB accessory, your application must have permission from your users.

Note If your application uses an intent filter to discover USB devices as they’re connected, it automatically receives permission if the user allows your application to handle the intent. If not, you must request permission explicitly in your application before connecting to the device.

Explicitly asking for permission might be necessary in some situations such as when your application enumerates USB devices that are already connected and then wants to communicate with one.

To explicitly obtain permission:

  1. Call UsbManager.requestPermission().

  2. The call to requestPermission() displays a dialog to the user asking for permission to connect to the accessory.

  3. The system generates a broadcast intent with a boolean EXTRA_PERMISSION_GRANTED extra indicating the user’s response.

The following sample code shows how to create a broadcast receiver to process the response:

private static final String ACTION_USB_PERMISSION = "com.android.example.USB_PERMISSION";

private final BroadcastReceiver mUsbReceiver = new BroadcastReceiver() {

    public void onReceive(Context context, Intent intent) {
        String action = intent.getAction();
        if (ACTION_USB_PERMISSION.equals(action)) {
            synchronized (this) {
            UsbAccessory accessory = (UsbAccessory) intent.getParcelableExtra(UsbManager.EXTRA_ACCESSORY);

                if (intent.getBooleanExtra(UsbManager.EXTRA_PERMISSION_GRANTED, false)) {
                    if (accessory != null) {
                      // Call method to set up accessory communication
                   }
                }
                else {
                    Log.d(TAG, "permission denied for accessory " + accessory);
                }
            }
        }
    }
};

To register the broadcast receiver, add this in your onCreate() method in your activity:

IntentFilter filter = new IntentFilter(ACTION_USB_PERMISSION);
registerReceiver(mUsbReceiver, filter);

To display the dialog that asks users for permission to connect to the device, call the UsbManager.requestPermission() method:

UsbManager mUsbManager = (UsbManager) getSystemService(Context.USB_SERVICE);
UsbAccessory accessory;
// ...
mPermissionIntent = PendingIntent.getBroadcast(this, 0, new Intent(ACTION_USB_PERMISSION), 0);
mUsbManager.requestPermission(accessory, mPermissionIntent);

      Communicating With an Accessory

You can communicate with the accessory by using the UsbManager to obtain a file descriptor that you can set up input and output streams to read and write data to descriptor.

UsbAccessory mAccessory;
ParcelFileDescriptor mFileDescriptor;
FileInputStream mInputStream;
FileOutputStream mOutputStream;

// ...

private void openAccessory() {
    Log.d(TAG, "openAccessory: " + accessory);
    mFileDescriptor = mUsbManager.openAccessory(mAccessory);
    if (mFileDescriptor != null) {
        FileDescriptor fd = mFileDescriptor.getFileDescriptor();
        mInputStream = new FileInputStream(fd);
        mOutputStream = new FileOutputStream(fd);
        Thread thread = new Thread(null, this, "AccessoryThread");
        thread.start();
    }
}

In the thread’s run() method, you can read and write to the accessory by using the FileInputStream or FileOutputStream objects.

When reading data from an accessory with a FileInputStream object, ensure that the buffer that you use is big enough to store the USB packet data.

Note At a lower level, the packets are 64 bytes for USB full-speed accessories and 512 bytes for USB high-speed accessories. The Android accessory protocol bundles the packets together for both speeds into one logical packet for simplicity.

      Terminating Communication With an Accessory

When you are done communicating with an accessory or if the accessory was detached, close the file descriptor that you opened by calling close().

To listen for detached events, create a broadcast receiver like below:

BroadcastReceiver mUsbReceiver = new BroadcastReceiver() {
    public void onReceive(Context context, Intent intent) {
        String action = intent.getAction();

        if (UsbManager.ACTION_USB_ACCESSORY_DETACHED.equals(action)) {
            UsbAccessory accessory = (UsbAccessory)intent.getParcelableExtra(UsbManager.EXTRA_ACCESSORY);
            if (accessory != null) {
                // call your method that cleans up and closes communication with the accessory
            }
        }
    }
};

Creating the broadcast receiver within the application, and not the manifest, allows your application to handle detached events only while it is running.

      The Android Open Accessory Development Kit

The Android Open Accessory Development Kit (ADK) provides an implementation of an Android USB accessory that is based on the Arduino open source electronics prototyping platform.

The main hardware and software components of the ADK include:

   Summary of USB Module

In this module, you learned about support for USB in Android. We explored how USB works, how to use USB Host mode, and how to use USB Accessories.

Android Tools and Debugging

   Objectives of Tools and Debugging Module

Android is a complex system, composed of many moving parts and layers. In this module, we will explore Android’s many tools and configuration options that will enable us to:

   Kernel Debugging

   Logging

   Stack Traces

   Debugging Native Code with GDB

  1. Map a port on our host machine to a port on the emulator or device that GDB will listen on: 

    $ adb forward tcp:5039 tcp:5039
    About Port Mapping

    Here, we use port 5039 because it is what gdbclient helper function uses (provided by build/envsetup.sh), although the "official" port for GDB is 2159, according IANA.

    When this is done, we’ll have 127.0.0.1:5039 on our host machine → 0.0.0.0:5039 on the emulator/device.

    Emulator also permits port-mapping via its management console (assuming it’s accessible via 5554):

    $ echo "redir add tcp:5039:5039" | nc localhost 5554

  2. Debug a new program or an existing process with the help of the GDB server:

    • Get GDB server to start a new process by specifying its path and arguments. For example: 
      $ adb shell gdbserver :5039 /system/bin/mrknlog &
      Note Here, we are starting a (custom) program (mrknlog) that prints the used/total size of /dev/log/main buffer and flushes it before exiting.
    • Get GDB server to attach to an existing process via its PID. For example: 
      $ pid mrknlogd
47
$ adb shell gdbserver :5039 --attach 47 &
Attached; pid = 47
Listening on port 5039
      Note Here, we are connecting to a (custom) daemon process (mrknlogd), which does exactly what mrknlog does, except repeatedly, while sleeping between executions. We assume mrknlogd is already running (e.g. started by init).
      Tip

      The shell function pid mrknlogd (provided by build/envsetup.sh) is equivalent to:

      $ adb shell ps | grep mrknlogd  | awk '{print $2}'
47

  3. Get the location of unstripped binaries: 

    $ export SYMBOLS=$ANDROID_PRODUCT_OUT/symbols
    Note

    Here, ANDROID_PRODUCT_OUT is set by the lunch command.

    Alternatively, we could also do:

    $ export SYMBOLS=`get_abs_build_var TARGET_OUT_UNSTRIPPED`
    About Debugger Symbols

    To save on (flash-memory) space, Android by default strips all of the symbols from its executables and shared libraries. While symbols (i.e. the debug info) are not needed at run-time, GDB uses the symbol table to resolve line numbers as well as function and variable names. We’d be lost trying to debug without these.

    So, what do we do?

    We could provide GDB with location of unstripped binaries (the direction we’ll take).

    Or, we could re-compile our binaries and instruct the build system not to strip the symbol information (as well as disable compiler optimizations), by placing the following in our Android.mk file(s):

    LOCAL_CFLAGS += -g -O0

    Replace LOCAL_CFLAGS with LOCAL_CPPFLAGS in case of C++.

  4. Connect with the debugger client (gdb) to the debugger server (gdbserver): 

    $ANDROID_TOOLCHAIN/arm-linux-androideabi-gdb \
  -ex "set solib-absolute-prefix $SYMBOLS" \
  -ex "set solib-search-path $SYMBOLS/system/lib:$SYMBOLS/system/lib/hw:$SYMBOLS/system/lib/ssl/engines" \
  -ex "target remote :5039" \
  --quiet \
  $SYMBOLS/system/bin/mrknlog
Reading symbols from /home/student/aosp/out/target/product/alpha/symbols/system/bin/mrknlog...done.
Remote debugging using :5039
Remote debugging from host 127.0.0.1
__dl__start () at bionic/linker/arch/arm/begin.S:35
35      mov r0, sp
(gdb)
    Note For x86 targets, run $ANDROID_EABI_TOOLCHAIN/i686-android-linux-gdb instead. Replace mrknlog with mrknlogd when attaching to our mrknlogd daemon.
    Tip Android ships with a gdbclient bash function (courtesy of build/envsetup.sh) that tries to automate all of these steps, though its usage is somewhat confusing.
    Tip You may find GDB cheatsheet handy.
    About the GDB Client

    Here we are running gdb client from the Android-provided arch-specific toolchain.

    Using the -ex command-line options, we are asking gdb to:

    1. Initialize the system root to the root of our un-stripped binaries

    2. Initialize the search path for shared objects to the locations of our unstripped shared libraries

    3. Connect to the GDB server on the previously mapped port (127.0.0.1:5039)

    Finally, we are also providing gdb with the unstripped version of the executable we are debugging.

  5. Start debugging:

    • Debugging a new program, starting from main(…): 
      (gdb) break main
Breakpoint 1 at 0x2a0004e0: file device/marakana/alpha/bin/mrknlog/mrknlog.c, line 7.
(gdb) continue
Continuing.

Breakpoint 1, main (argc=1, argv=0xbee96cb4) at device/marakana/alpha/bin/mrknlog/mrknlog.c:7
7   int main (int argc, char* argv[]) {
(gdb) list
2   #include <string.h>
3   #include <errno.h>
4
5   #include <mrknlog.h>
6
7   int main (int argc, char* argv[]) {
8       int usedSize = mrkn_get_used_log_size();
9       int totalSize = mrkn_get_total_log_size();
10      if (totalSize >= 0 && usedSize >= 0) {
11          if (mrkn_flush_log() == 0) {
(gdb) b 11
Breakpoint 2 at 0x2a0004fe: file device/marakana/alpha/bin/mrknlog/mrknlog.c, line 11.
(gdb) c
Continuing.

Breakpoint 2, main (argc=<optimized out>, argv=<optimized out>)
    at device/marakana/alpha/bin/mrknlog/mrknlog.c:11
11          if (mrkn_flush_log() == 0) {
(gdb) print usedSize
$1 = 223
(gdb) bt
#0  main (argc=<optimized out>, argv=<optimized out>) at device/marakana/alpha/bin/mrknlog/mrknlog.c:11
…
    • Debugging an existing process (mrknlogd): 
      …
Reading symbols from /home/student/aosp/out/target/product/alpha/symbols/system/bin/mrknlogd...done.
Remote debugging using :5039
Remote debugging from host 127.0.0.1
warning: .dynamic section for "/home/student/aosp/out/target/product/alpha/symbols/system/bin/linker" is not at the expected address (wrong library or version mismatch?)
nanosleep () at bionic/libc/arch-arm/syscalls/nanosleep.S:10
10      ldmfd   sp!, {r4, r7}
(gdb) bt
#0  nanosleep () at bionic/libc/arch-arm/syscalls/nanosleep.S:10
#1  0x40043294 in sleep (seconds=<optimized out>) at bionic/libc/unistd/sleep.c:45
#2  0x2a0005e0 in main (argc=<optimized out>, argv=<optimized out>)
    at device/marakana/alpha/bin/mrknlogd/mrknlogd.c:28
(gdb) frame 2
#2  0x2a0005e0 in main (argc=<optimized out>, argv=<optimized out>)
    at device/marakana/alpha/bin/mrknlogd/mrknlogd.c:28
28        sleep(frequency);
(gdb) print frequency
$1 = 60
(gdb) set frequency=10
(gdb) print frequency
$2 = 60
(gdb) break 27
Breakpoint 1 at 0x2a0005da: file device/marakana/alpha/bin/mrknlogd/mrknlogd.c, line 27.
(gdb) c
Continuing.

Breakpoint 1, main (argc=<optimized out>, argv=<optimized out>)
    at device/marakana/alpha/bin/mrknlogd/mrknlogd.c:28
28        sleep(frequency);
(gdb) print usedSize
$6 = 415
(gdb) detach
Detaching from process 47
Ending remote debugging.
(gdb) quit
$
    • Debugging a shared library within an existing Java (Dalvik) process (com.marakana.android.logservice): 
      $ adb shell ps | grep system.*com.marakana.android.logservice | awk '{print $2}'
300
$ adb shell gdbserver :5039 --attach 300 &
Attached; pid = 300
Listening on port 5039
$ $ANDROID_TOOLCHAIN/arm-linux-androideabi-gdb \
  -ex "set solib-absolute-prefix $SYMBOLS" \
  -ex "set solib-search-path $SYMBOLS/system/lib:$SYMBOLS/system/lib/hw:$SYMBOLS/system/lib/ssl/engines" \
  -ex "set breakpoint pending on" \
  -ex "target remote :5039" \
  --quiet \
  $SYMBOLS/system/bin/app_process
Reading symbols from /home/student/aosp/out/target/product/alpha/symbols/system/bin/app_process...done.
Remote debugging using :5039
Remote debugging from host 127.0.0.1
warning: .dynamic section for "/home/student/aosp/out/target/product/alpha/symbols/system/bin/linker" is not at the expected address (wrong library or version mismatch?)
epoll_wait () at bionic/libc/arch-arm/syscalls/epoll_wait.S:10
10      ldmfd   sp!, {r4, r7}
(gdb) b device/marakana/alpha/lib/libmrknlog/libmrknlog.c:22
Breakpoint 1 at 0x49f96504: file device/marakana/alpha/lib/libmrknlog/libmrknlog.c, line 22.
(gdb) c
Continuing.
[New Thread 1171]
[Switching to Thread 1171]

Breakpoint 1, ioctl_log (mode=<optimized out>, request=44546)
    at device/marakana/alpha/lib/libmrknlog/libmrknlog.c:24
24  }
(gdb) print request
$1 = 44546
(gdb) detach
(gdb) quit
      About Debugging Dalvik Processes

      Debugging native-code in any Dalvik-based (or Zygote-forked) process requires that we start from system/bin/app_process, because that’s what zygote first runs. In order to reference code in shared libraries (yet to be loaded), we need enable pending break points.

      We can either do this via -ex "set breakpoint pending on" command-line switch, or in GDB when it starts:

      (gdb) set breakpoint pending on

      Debugging Java code will be covered later.

    • Debugging a shared library within System Server 
      $ pid system_server
307
$ adb shell gdbserver :5039 --attach 307 &
Attached; pid = 307
Listening on port 5039
$ $ANDROID_TOOLCHAIN/arm-linux-androideabi-gdb \
  -ex "set solib-absolute-prefix $SYMBOLS" \
  -ex "set solib-search-path $SYMBOLS/system/lib:$SYMBOLS/system/lib/hw:$SYMBOLS/system/lib/ssl/engines:$SYMBOLS/system/vendor/lib/hw:$ANDROID_PRODUCT_OUT/obj/lib" \
  -ex "set breakpoint pending on" \
  -ex "target remote :5039" \
  --quiet \
  $SYMBOLS/system/bin/app_process
Reading symbols from /Volumes/Android/aosp-4.1/out/target/product/maguro/symbols/system/bin/app_process...done.
Remote debugging using :5039
Remote debugging from host 127.0.0.1
warning: .dynamic section for "/Volumes/Android/aosp-4.1/out/target/product/maguro/symbols/system/bin/linker" is not at the expected address (wrong library or version mismatch?)
__ioctl () at bionic/libc/arch-arm/syscalls/__ioctl.S:9
9       swi     #0
(gdb) break hardware/libhardware_legacy/vibrator/vibrator.c:61
Breakpoint 1 at 0x400ecc3c: file hardware/libhardware_legacy/vibrator/vibrator.c, line 61.
(gdb) c
Continuing.

Breakpoint 1, sendit (timeout_ms=20) at hardware/libhardware_legacy/vibrator/vibrator.c:61
61      close(fd);
(gdb) bt
#0  sendit (timeout_ms=20) at hardware/libhardware_legacy/vibrator/vibrator.c:61
#1  0x400ecc8c in vibrator_on (timeout_ms=20) at hardware/libhardware_legacy/vibrator/vibrator.c:69
#2  0x56f90d3c in android::vibratorOn (env=0x5c514d30, clazz=0x40200009, timeout_ms=20) at frameworks/base/services/jni/com_android_server_VibratorService.cpp:40
#3  0x4073de34 in dvmPlatformInvoke () at dalvik/vm/arch/arm/CallEABI.S:258
#4  0x4076d086 in dvmCallJNIMethod (args=0x5bbe1794, pResult=0x58000400, method=0x572856d0, self=0x580003f0) at dalvik/vm/Jni.cpp:1155
…
#23 0x407743d6 in interpThreadStart (arg=0x580003f0) at dalvik/vm/Thread.cpp:1538
#24 0x40029bb4 in __thread_entry (func=0x40774335 <interpThreadStart(void*)>, arg=0x580003f0, tls=<optimized out>) at bionic/libc/bionic/pthread.c:217
#25 0x4002930c in pthread_create (thread_out=0x5c3f1e28, attr=0x5e2bed70, start_routine=0x40774335 <interpThreadStart(void*)>, arg=0x580003f0) at bionic/libc/bionic/pthread.c:356
#26 0x00000000 in ?? ()
(gdb) frame 2
#2  0x56f90d3c in android::vibratorOn (env=0x5c514d30, clazz=0x40200009, timeout_ms=20) at frameworks/base/services/jni/com_android_server_VibratorService.cpp:40
40      vibrator_on(timeout_ms);
(gdb) list
35  }
36
37  static void vibratorOn(JNIEnv *env, jobject clazz, jlong timeout_ms)
38  {
39      // ALOGI("vibratorOn\n");
40      vibrator_on(timeout_ms);
41  }
42
43  static void vibratorOff(JNIEnv *env, jobject clazz)
44  {
(gdb) print timeout_ms
$1 = 20
(gdb) detach
(gdb) quit
      Note

      In this case, we disabled:

      Otherwise, we would have had harder time walking through the code we were interested in.
      About Debugging with Proprietary Binaries

      When debugging on real hardware devices, we should also include proprietary binaries (mostly HAL libraries) into our solib-search-path.

      These binaries are not under $ANDROID_PRODUCT_OUT/symbols/*, because they come pre-built.

      All we need to do is append $SYMBOLS/system/vendor/lib/hw:$ANDROID_PRODUCT_OUT/obj/lib to solib-search-path.

      About Debugging Multithreaded Processes

      Most processes on Android are multi-threaded, but GDB can only step through one thread at a time. In order to prevent (suspend) threads from running while stepping through code, we can:

      (gdb) set scheduler-locking on

      We just have to remember to either continue or disable the lock, in order to allow for normal execution. Otherwise, we could cause a dead-lock.

      (gdb) set scheduler-locking off

      For more info, see Debugging Programs with Multiple Threads

   Debugging Android Applications

   Debugging Existing Java Applications/Processes

      Debugging ActivityManagerService Example

$ adb forward tcp:8700 jdwp:$(pid system_server)
jdb -attach 127.0.0.1:8700
Set uncaught java.lang.Throwable
Set deferred uncaught java.lang.Throwable
Initializing jdb ...
> stop in com.android.server.am.ActivityManagerService.updateOomAdjLocked()
Set breakpoint com.android.server.am.ActivityManagerService.updateOomAdjLocked()
> stop in com.android.server.am.ActivityManagerService.updateOomAdjLocked(
Breakpoint hit: "thread=12 android.server.ServerThread", com.android.server.am.ActivityManagerService.updateOomAdjLocked(), line=14,705 bci=0

> stop at com.android.server.am.ActivityManagerService:14731
Set breakpoint com.android.server.am.ActivityManagerService:14731
12 android.server.ServerThread[1] cont
12 android.server.ServerThread[1] where
  [1] com.android.server.am.ActivityManagerService.updateOomAdjLocked (ActivityManagerService.java:14,731)
  [2] com.android.server.am.BroadcastQueue.deliverToRegisteredReceiverLocked (BroadcastQueue.java:418)
  [3] com.android.server.am.BroadcastQueue.processNextBroadcast (BroadcastQueue.java:634)
  [4] com.android.server.am.ActivityManagerService.finishReceiver (ActivityManagerService.java:13,281)
  [5] android.content.BroadcastReceiver$PendingResult.sendFinished (BroadcastReceiver.java:417)
  [6] android.content.BroadcastReceiver$PendingResult.finish (BroadcastReceiver.java:393)
  [7] android.app.LoadedApk$ReceiverDispatcher$Args.run (LoadedApk.java:772)
  [8] android.os.Handler.handleCallback (Handler.java:615)
  [9] android.os.Handler.dispatchMessage (Handler.java:92)
  [10] android.os.Looper.loop (Looper.java:137)
  [11] com.android.server.ServerThread.run (SystemServer.java:875)
12 android.server.ServerThread[1] print numSlots
numSlots = 7
12 android.server.ServerThread[1] print factor
factor = 2
12 android.server.ServerThread[1] print i
i = 21
12 android.server.ServerThread[1] quit
Tip Recall that pid is a function exposed by build/envsetup.sh.

   Digging into System Services

Legal Disclaimer

INFORMATION IN THIS DOCUMENT IS PROVIDED “AS IS”. NO LICENSE, EXPRESS OR IMPLIED, BY ESTOPPEL OR OTHERWISE, TO ANY INTELLECTUAL PROPERTY RIGHTS IS GRANTED BY THIS DOCUMENT. MARAKANA ASSUMES NO LIABILITY WHATSOEVER AND MARAKANA DISCLAIMS ANY EXPRESS OR IMPLIED WARRANTY, RELATING TO THIS INFORMATION INCLUDING LIABILITY OR WARRANTIES RELATING TO FITNESS FOR A PARTICULAR PURPOSE, MERCHANTABILITY, OR INFRINGEMENT OF ANY PATENT, COPYRIGHT OR OTHER INTELLECTUAL PROPERTY RIGHT.

Other names and brands may be claimed as the property of others.

Copyright © 2012 Marakana Inc.

← →

/

#